Unified Communications or "UC" is a high level suite of services. Those services are provided by combining several products. The underlying products can be very disperse and very complex. Each product employs processes and architectures that fit the operational environment. It is no longer feasible for a single individual to understand the complete system end to end. Effective troubleshooting of UC requires strong troubleshooting skills combined with detailed knowledge of underlying products and protocols combined with good communication skills. Neither Rome nor a fully functional enterprise contact center were built in a single day.
Cisco provides only a part of the stack; Partners, Operators, Users, and 3rd party applications each make valuable contributions toward the overall service. Cisco may not understand the CVP call flow designed to address your business process the same way that you may not understand CUCM versioning. Cisco needs to work with you the same way CUCM and CVP work together to deliver calls. Actually Cisco and you need to work together better than the products do. Products stop at protocol interaction and protocols rarely, if ever, enable your complete business process. We can work together by adopting similar practices starting with a common approach to troubleshooting.
A Common and Effective Approach to Troubleshooting
"A problem well stated is half solved."
Charles F. Kettering American engineer, inventor of the electric starter, 1876-1958
Before raising a case with the Cisco Technical Assistance Center, you can execute a few simple steps to enable a speedier solution to your problem, or even prevent the call altogether.
- STATE the expected behavior. That is, how will we know that the problem is resolved and conforming to expectations?
- OBSERVE the actual, undesired system behavior. How is the current observed behavior deviating from what is expected?
- DETERMINE WHEN it started happening. Did any other event occur at that time? What changed?
- LOCALIZE where is it happening. What parts are exhibiting the deviation and what parts are working as expected?
- COLLECT the facts. When a problem happens, there are a lot of rumors, guesses and projections. Focus on repeatable documented events. Document any deviation from what is expected.
- LIST what has already been tried to resolve the problem, with the observed results of each action. Sometimes, troubleshooting steps induce secondary problems that mask the original problem. Track all diagnostic steps.
- CORRELATE information. Are there any similar systems or processes you would expect to be failing that are not? Focusing on and comparing to working systems and "known good" systems can help isolate an issue
Answering these questions for yourself is the first step. Sharing those answers with all parties is the critical, and often overlooked, second step. These seven points are an introduction to a much more lengthy evolved process that is proven effective. Cisco TAC uses Kepner Tregoe (Wikipedia entry) as a starting point for consistency in our world wide support operations. Cisco also has strong ties to the Internet community which flavor our processes. Here are a few good starting points for further information:
- CCVP course titled "Troubleshooting Cisco Unified Communications Systems (TUC)" (pre-reqs required)
- UC System Troubleshooting Methodology
- UC System Troubleshooting Daily Operations
With situation analysis complete you are ready to dig into technical details of the deviation.
Matrix of Technical References
Reference below help to establish a baseline of troubleshooting knowledge. These are conepts and their respective implementation in each of of the products that make up Unfied Communications.
By starting with a high level process, answering questions, communicating, and then following concepts into product specific details it is possible to support the series of products that comprise UC.
In no particular order thanks to Phillip Remaker, Terry Mar, Shane Kirby, Brendan Shank, Hitesh Patel, Gonzalo Salgueiro, Manit Suphavadeprasit, Scott Page, and Pat Hayes for their contributions.