Troubleshooting Unified Communications


Tue, 08/23/2011 - 10:29
Apr 9th, 2010
User Badges:
  • Cisco Employee,

Unified Communications or "UC" is a high level suite of services.  Those services are provided by combining several products.  The underlying products can be very disperse and very complex.  Each product employs processes and architectures that fit the operational environment.   It is no longer feasible for a single individual to understand the complete system end to end.  Effective troubleshooting of UC requires strong troubleshooting skills combined with detailed knowledge of underlying products and protocols combined with good communication skills.  Neither Rome nor a fully functional enterprise contact center were built in a single day.

Cisco provides only a part of the stack; Partners, Operators, Users, and 3rd party applications each make valuable contributions toward the overall service.  Cisco may not understand the CVP call flow designed to address your business process the same way that you may not understand CUCM versioning.  Cisco needs to work with you the same way CUCM and CVP work together to deliver calls.  Actually Cisco and you need to work together better than the products do.  Products stop at protocol interaction and protocols rarely, if ever, enable your complete business process.  We can work together by adopting similar practices starting with a common approach to troubleshooting.

A Common and Effective Approach to Troubleshooting

"A problem well stated is half solved."

Charles F. Kettering American engineer, inventor of the electric starter, 1876-1958

Before raising a case with the Cisco Technical Assistance Center, you can execute a few simple steps to enable a speedier solution to your problem, or even prevent the call altogether.

  1. STATE the expected behavior. That is, how will we know that the problem is resolved and conforming to expectations?   
  2. OBSERVE the actual, undesired system behavior. How is the current observed behavior deviating from what is expected?   
  3. DETERMINE WHEN it started happening. Did any other event occur at that time? What changed?   
  4. LOCALIZE where is it happening. What parts are exhibiting the deviation and what parts are working as expected?   
  5. COLLECT the facts. When a problem happens, there are a lot of rumors, guesses and projections. Focus on repeatable documented events. Document any deviation from what is expected.   
  6. LIST what has already been tried to resolve the problem, with the observed results of each action. Sometimes, troubleshooting steps induce secondary problems that mask the original problem. Track all diagnostic steps.   
  7. CORRELATE information. Are there any similar systems or processes you would expect to be failing that are not? Focusing on and comparing to working systems and "known good" systems can help isolate an issue

Answering these questions for yourself is the first step.  Sharing those answers with all parties is the critical, and often overlooked, second step.  These seven points are  an introduction to a much more lengthy evolved process that is proven effective.  Cisco TAC uses Kepner Tregoe (Wikipedia entry) as a starting point for consistency in our world wide support operations.  Cisco also has strong ties to the Internet community which flavor our processes.  Here are a few good starting points for further information:

With situation analysis complete you are ready to dig into technical details of the deviation.

Matrix of Technical References 

Reference below help to establish a baseline of troubleshooting knowledge.  These are conepts and their respective implementation in each of of the products that make up Unfied Communications.


Multiservice Voice GatewaysMultiservice Solutions for service providersIPCC EnterpriseIPCC ExpressUnified Communications Manager(CallManager)Emergency Responder and PhonesUnity and Unity ConnectionUnified Presence
Hot Issues RSSVoice Gateways, CME, CUE
Cisco Customer Voice Portal (CVP) and Contact Center Enterprise (UCCE)IPCCXCMEndpoints, VOSUnity, Unity ConnectionCisco Unified Presence, Endpoints
What traces to use and how to get them
Multiservice Voice Debug Lookup
Tracing Levels for Troubleshooting UCCE Issues
Set Up Cisco CallManager Traces for Cisco Technical Support,
CUCM Serviceability Guide
VOS CLI Reference

How to Set, Configure, and Collect Unity Diagnostic Traces with Cisco UDTSet Up CUP and CUPC Traces for Cisco Technical Support
Trace Viewers and Decoders

Triple Combo Tool,
Error Message Decoder

Mini_parse for PGW2200

Troubleshooting Tips for Unified CVP 7.x ,

Troubleshooting Tips for Unified ICM/CCE & Hosted 7.5

UCCE Error Codes

CAD Error Code DictionaryTranslator X,
Triple Combo Tool,

Unity Error Lookup Tool (no longer updated, but useful for Unity 4.x)

Unity Diagnostic Viewer

Fax, Modem, and Text for IP Telephony

Troubleshooting Cisco IP Telephony

Unity Deployment and Solutions Guide

Unity Fundamentals

Deploying Cisco Unified Presence

Community Sites

DocWiki for UCCX Troubleshooting ,

Cisco Support Community for Contact Center

Cisco Unity ToolsDocWiki for CUPS
DSP Calculator
Sizing and Resources Calculator - Use CVP and General Resources Tab

UC Capacity Calculators

CM Capacity Tool

Compatibility Matrix
Cisco Unified CME and Cisco IOS Software Version Compatibility Matrix
Cisco Unity Express Compatibility Matrix

PGW Compatibility Matrix

BTS10200 Compatibility Matrix

Cisco Unified Contact Center Enterprise (Unified CCE) Software Compatibility Guide

CVP Compatibility Guide

Compatibility Guides for UCCX and IPIVRUC Compatibility Matricies (hardware and software)

Unity/CUCM Compatibility Matrix

UC/CUCM Compatibility Matrix

Unity Subscriber Workstation Compatibility Matrix

UC User Workstation Compatibility Matrix

CUP 7.x HW and SW Compatibility Matrix

Random Useful Pages

Cisco IOS Voice Troubleshooting and Monitoring Guide

Understanding Inbound and Outbound Dial Peers Matching on IOS Platforms

Analog Voice Port Best Match Impedance Setting Choice

Identify, Supersede, and Maintain Compatible DSP Firmware Version Levels on C5510 DSPs

DSP on NM-HDV2 Functionality Verification for 2600XM/2691/2800/3700/3800 Platforms

Troubleshooting the DSP on NM-HDV for Cisco 2600/3600/VG200 Series Routers

CallManager and IOS Gateway DSP Farm Configuration Example

Troubleshooting and Debugging VoIP Call Basics

Cisco CallManager Express Security Guide to Best Practices

PGW 2200 Softswitch: Collect Troubleshooting Information for TAC Service Requests

HSI:  Collect Troubleshooting Information for TAC Service Requests

PGW2200: Troubleshooting Mute Calls

UCCE Troubleshooting


All Aboard the Express - Partner Portal for UCCX

CUCM Tech Notes,

CUCM to PBX Interoperability Guides

UC System Docs

CER Troubleshooting Guide,

Reconfiguration and Upgrade Guide for Cisco Unity Release 5.x (With Microsoft Exchange)

Reconfiguration and Upgrade Guide for Cisco Unity Connection Release 7.x

UC Developer Support

Unity Netpro Forum

Troubleshoot CUPS and CUPC

Third Party Integration with CUP

General Tools

Partner Central
SRND - design guides
Cisco Voice Interoperability Portal
Cisco Bug Toolkit - bug searches, recently improved searches
Cisco Support Forums - documents and discussions
Cisco-VOIP forum, Cisco-VOIP archive - a public mailer where customers, partners, and Cisco discuss all Cisco UC products
Collaborative Support Community for SMB

Unified Communications Partner Community - tech docs, forums, videos, blogs for use by Partners

Cisco Community Central's "Unified Communications" Community - tech docs, forums, videos, blogs for use by all

By starting with a high level process, answering questions, communicating, and then following concepts into product specific details it is possible to support the series of products that comprise UC.


In no particular order thanks to Phillip Remaker, Terry Mar, Shane Kirby, Brendan Shank, Hitesh Patel, Gonzalo Salgueiro, Manit Suphavadeprasit, Scott Page, and Pat Hayes for their contributions.

Tommer Catlin Mon, 04/12/2010 - 08:22
User Badges:
  • Green, 3000 points or more

Pretty much the holy grail of Cisco shortcuts.    Nice job everyone!

Rob Huffman Wed, 05/05/2010 - 05:30
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 IP Telephony, Unified Communications

I agree with TCAT ...this is sweet! Thanks to Wes and all involved

Wes Sisk Wed, 05/05/2010 - 05:34
User Badges:
  • Cisco Employee,

I will pass that along to the team.  If you have ideas of valuable content let us known.


This Document

Related Content