Troubleshooting Unified Communications

Document

Apr 9, 2010 8:18 AM
Apr 9th, 2010

Unified Communications or "UC" is a high level suite of services.  Those services are provided by combining several products.  The underlying products can be very disperse and very complex.  Each product employs processes and architectures that fit the operational environment.   It is no longer feasible for a single individual to understand the complete system end to end.  Effective troubleshooting of UC requires strong troubleshooting skills combined with detailed knowledge of underlying products and protocols combined with good communication skills.  Neither Rome nor a fully functional enterprise contact center were built in a single day.

Cisco provides only a part of the stack; Partners, Operators, Users, and 3rd party applications each make valuable contributions toward the overall service.  Cisco may not understand the CVP call flow designed to address your business process the same way that you may not understand CUCM versioning.  Cisco needs to work with you the same way CUCM and CVP work together to deliver calls.  Actually Cisco and you need to work together better than the products do.  Products stop at protocol interaction and protocols rarely, if ever, enable your complete business process.  We can work together by adopting similar practices starting with a common approach to troubleshooting.

A Common and Effective Approach to Troubleshooting

"A problem well stated is half solved."

Charles F. Kettering American engineer, inventor of the electric starter, 1876-1958

Before raising a case with the Cisco Technical Assistance Center, you can execute a few simple steps to enable a speedier solution to your problem, or even prevent the call altogether.

  1. STATE the expected behavior. That is, how will we know that the problem is resolved and conforming to expectations?   
  2. OBSERVE the actual, undesired system behavior. How is the current observed behavior deviating from what is expected?   
  3. DETERMINE WHEN it started happening. Did any other event occur at that time? What changed?   
  4. LOCALIZE where is it happening. What parts are exhibiting the deviation and what parts are working as expected?   
  5. COLLECT the facts. When a problem happens, there are a lot of rumors, guesses and projections. Focus on repeatable documented events. Document any deviation from what is expected.   
  6. LIST what has already been tried to resolve the problem, with the observed results of each action. Sometimes, troubleshooting steps induce secondary problems that mask the original problem. Track all diagnostic steps.   
  7. CORRELATE information. Are there any similar systems or processes you would expect to be failing that are not? Focusing on and comparing to working systems and "known good" systems can help isolate an issue

Answering these questions for yourself is the first step.  Sharing those answers with all parties is the critical, and often overlooked, second step.  These seven points are  an introduction to a much more lengthy evolved process that is proven effective.  Cisco TAC uses Kepner Tregoe (Wikipedia entry) as a starting point for consistency in our world wide support operations.  Cisco also has strong ties to the Internet community which flavor our processes.  Here are a few good starting points for further information:

With situation analysis complete you are ready to dig into technical details of the deviation.

Matrix of Technical References 

Reference below help to establish a baseline of troubleshooting knowledge.  These are conepts and their respective implementation in each of of the products that make up Unfied Communications.


Multiservice
IPCC
CUCM
Apps


Multiservice Voice GatewaysMultiservice Solutions for service providersIPCC EnterpriseIPCC ExpressUnified Communications Manager(CallManager)Emergency Responder and PhonesUnity and Unity ConnectionUnified Presence
Hot Issues RSSVoice Gateways, CME, CUE
Cisco Customer Voice Portal (CVP) and Contact Center Enterprise (UCCE)IPCCXCMEndpoints, VOSUnity, Unity ConnectionCisco Unified Presence, Endpoints
What traces to use and how to get them
Multiservice Voice Debug Lookup
Tracing Levels for Troubleshooting UCCE Issues
Set Up Cisco CallManager Traces for Cisco Technical Support,
CUCM Serviceability Guide
,
VOS CLI Reference

How to Set, Configure, and Collect Unity Diagnostic Traces with Cisco UDTSet Up CUP and CUPC Traces for Cisco Technical Support
Trace Viewers and Decoders

Triple Combo Tool,
Error Message Decoder

Mini_parse for PGW2200

Troubleshooting Tips for Unified CVP 7.x ,

Troubleshooting Tips for Unified ICM/CCE & Hosted 7.5

UCCE Error Codes

CAD Error Code DictionaryTranslator X,
Triple Combo Tool,
VLT

Unity Error Lookup Tool (no longer updated, but useful for Unity 4.x)

Unity Diagnostic Viewer



Books
Fax, Modem, and Text for IP Telephony


Troubleshooting Cisco IP Telephony

Unity Deployment and Solutions Guide

Unity Fundamentals


Deploying Cisco Unified Presence


Community Sites
UC500.com

DocWiki for UCCX Troubleshooting ,

Cisco Support Community for Contact Center

Cisco-VOIP
Cisco Unity ToolsDocWiki for CUPS
Calculators
DSP Calculator
Sizing and Resources Calculator - Use CVP and General Resources Tab

UC Capacity Calculators


CM Capacity Tool





Compatibility Matrix
Cisco Unified CME and Cisco IOS Software Version Compatibility Matrix
Cisco Unity Express Compatibility Matrix

PGW Compatibility Matrix

BTS10200 Compatibility Matrix

Cisco Unified Contact Center Enterprise (Unified CCE) Software Compatibility Guide

CVP Compatibility Guide

Compatibility Guides for UCCX and IPIVRUC Compatibility Matricies (hardware and software)

Unity/CUCM Compatibility Matrix

UC/CUCM Compatibility Matrix

Unity Subscriber Workstation Compatibility Matrix

UC User Workstation Compatibility Matrix


CUP 7.x HW and SW Compatibility Matrix


Random Useful Pages

Cisco IOS Voice Troubleshooting and Monitoring Guide


Understanding Inbound and Outbound Dial Peers Matching on IOS Platforms


Analog Voice Port Best Match Impedance Setting Choice


Identify, Supersede, and Maintain Compatible DSP Firmware Version Levels on C5510 DSPs


DSP on NM-HDV2 Functionality Verification for 2600XM/2691/2800/3700/3800 Platforms


Troubleshooting the DSP on NM-HDV for Cisco 2600/3600/VG200 Series Routers


CallManager and IOS Gateway DSP Farm Configuration Example


Troubleshooting and Debugging VoIP Call Basics


Cisco CallManager Express Security Guide to Best Practices

PGW 2200 Softswitch: Collect Troubleshooting Information for TAC Service Requests

HSI:  Collect Troubleshooting Information for TAC Service Requests

PGW2200: Troubleshooting Mute Calls

UCCE Troubleshooting

Guide

All Aboard the Express - Partner Portal for UCCX

CUCM Tech Notes,

CUCM to PBX Interoperability Guides

UC System Docs

CER Troubleshooting Guide,

Reconfiguration and Upgrade Guide for Cisco Unity Release 5.x (With Microsoft Exchange)

Reconfiguration and Upgrade Guide for Cisco Unity Connection Release 7.x

UC Developer Support

Unity Netpro Forum


Troubleshoot CUPS and CUPC


Third Party Integration with CUP


General Tools

Partner Central
SRND - design guides
Cisco Voice Interoperability Portal
Cisco Bug Toolkit - bug searches, recently improved searches
Cisco Support Forums - documents and discussions
Cisco-VOIP forum, Cisco-VOIP archive - a public mailer where customers, partners, and Cisco discuss all Cisco UC products
Collaborative Support Community for SMB

Unified Communications Partner Community - tech docs, forums, videos, blogs for use by Partners

Cisco Community Central's "Unified Communications" Community - tech docs, forums, videos, blogs for use by all


By starting with a high level process, answering questions, communicating, and then following concepts into product specific details it is possible to support the series of products that comprise UC.

References

In no particular order thanks to Phillip Remaker, Terry Mar, Shane Kirby, Brendan Shank, Hitesh Patel, Gonzalo Salgueiro, Manit Suphavadeprasit, Scott Page, and Pat Hayes for their contributions.

Average Rating: 5 (3 ratings)

Comments

Actions

Login or Register to take actions

This Document

Posted April 9, 2010 at 8:18 AM
Stats:
Comments:3 Avg. Rating:5
Views:4316 Contributors:3
Shares:0

Related Content