08-20-2010 10:08 AM - edited 03-16-2019 12:24 AM
Hi All,
I found one weird issue in our T1. We have one call center which has one number the customers can call in. I found out two calls which already ended several minutes ago. But they are still shown under the "show isdn active". They occupied the T1 slots. I called those two callers and they said they already hang up several minutes ago. At that time, the calls still show in active status.
We have a call queue setup in UCCX server. Is there any possiblity that the call ends actually but the queue still hold it for some time?
Thanks for your help!!!
Lou
Solved! Go to Solution.
08-24-2010 07:03 AM
(C3845-SPSERVICESK9-M), Version 12.4(22)T. Actually the calls will not hang there forever. I noticed that they usually only hang there for less than one minute.
That sounds like the media inactivity timer may be popping. Do you have that configured under the 'gateway' sub-configuration? Ideally, the call should have a disconnect coming in from one of the sides.
If you can get some debugs during one of these scenarios, we can take a look. If this is h323, you can also get 'sh h323 gateway cause' and see which value increments when the call drops.
debug voip ccapi inout
debug isdn q931 (assuming this is a PRI. Use 'debug vpm sig' if it isn't a PRI)
debug h225 asn1 (if this is h323)
debug ccsip mess (if this is SIP)
Collect debugs as follows.
Router(config)# service sequence
Router(config)# service timestamps debug datetime msec
Router(config)# logging buffered 10000000 7
Router(config)# no logging con
Router(config)# no logging mon
Router(config)# voice iec syslog
Router# term len 0
Router# sh logg
Note ANI/DNIS of a call with the issue.
09-10-2010 01:23 PM
Oh so the call isn't eternally stuck?
The problem is upstream of the gaetway, then. The disconnect doesn't come from the H323 peer (CM I presume?) until 10:20:
7381844: Sep 10 10:20:20.712 CDT: H225.0 INCOMING PDU ::=
value H323_UserInformation ::=
{
h323-uu-pdu
{
h323-message-body releaseComplete :
{
protocolIdentifier { 0 0 8 2250 0 5 }
callIdentifier
{
guid 'B54316E1BC2311DF901EFD8A79705EB9'H
}
}
h245Tunneling FALSE
}
}
7381845: Sep 10 10:20:20.712 CDT: //1652696/B5427AB9991B/CCAPI/cc_api_call_disconnected:
Cause Value=16, Interface=0x68589F00, Call Id=1652696
And then the call is torn down on ISDN right after that:
7381858: Sep 10 10:20:20.732 CDT: H225.0 OUTGOING PDU ::=
value H323_UserInformation ::=
{
h323-uu-pdu
{
h323-message-body releaseComplete :
{
protocolIdentifier { 0 0 8 2250 0 4 }
callIdentifier
{
guid 'B54316E1BC2311DF901EFD8A79705EB9'H
}
}
h245Tunneling FALSE
}
}
7381859: Sep 10 10:20:20.732 CDT: H225.0 OUTGOING ENCODE BUFFER::= 2580060008914A000411001100B54316E1BC2311DF901EFD8A79705EB910800100
7381860: Sep 10 10:20:20.732 CDT:
7381861: Sep 10 10:20:20.732 CDT: //1652696/B5427AB9991B/CCAPI/cc_api_call_disconnect_done:
Disposition=0, Interface=0x68589F00, Tag=0x0, Call Id=1652696,
Call Entry(Disconnect Cause=16, Voice Class Cause Code=0, Retry Count=0)
7381862: Sep 10 10:20:20.732 CDT: //1652696/B5427AB9991B/CCAPI/cc_api_call_disconnect_done:
Call Disconnect Event Sent
7381863: Sep 10 10:20:20.732 CDT: //-1/xxxxxxxxxxxx/CCAPI/cc_free_feature_vsa:
7381864: Sep 10 10:20:20.732 CDT: :cc_free_feature_vsa freeing 6CA41DC0
7381865: Sep 10 10:20:20.732 CDT: //-1/xxxxxxxxxxxx/CCAPI/cc_free_feature_vsa:
7381866: Sep 10 10:20:20.732 CDT: vsacount in free is 37
7381867: Sep 10 10:20:20.736 CDT: %ISDN-6-DISCONNECT: Interface Serial0/0/0:6 disconnected from 3147502946 , call lasted 880 seconds
7381868: Sep 10 10:20:20.736 CDT: ISDN Se0/0/0:23 Q931: TX -> DISCONNECT pd = 8 callref = 0x82FE
Cause i = 0x8090 - Normal call clearing
7381869: Sep 10 10:20:20.768 CDT: ISDN Se0/0/0:23 Q931: RX <- RELEASE pd = 8 callref = 0x02FE
7381870: Sep 10 10:20:20.768 CDT: ISDN Se0/0/0:23 Q931: TX -> RELEASE_COMP pd = 8 callref = 0x82FE
You need to investigate why the release complete transmission from the node across from dial-peer 100 is delays by 12 minutes. Detailed CM traces are the way to go.
08-20-2010 10:23 AM
I wouldn't think so...
Any way you could catch these hanging calls with a debug isdn q931?
08-20-2010 10:26 AM
Thanks. Let me try debugging this.
08-20-2010 10:41 AM
Ok. And can you provide any identifying information about the offending calls? Any patterns from ANI/DNIS evident? Can you recreate the issue at will?
08-20-2010 10:44 AM
I can't recreat issue at will. I must wait and compare the T1 active calls to cisco supervisor desktop to find out which calls are stuck in T1 which actullay already end. Really hard to troubleshoot.
08-23-2010 09:41 AM
Are you using a survivability.tcl file on any of these inbound calls? What version of IOS are you running on your inbound gateway?
08-23-2010 03:02 PM
No. I'm not running any script in the gateway. version is:
(C3845-SPSERVICESK9-M), Version 12.4(22)T
Thanks!
08-23-2010 10:30 AM
Common IOS bug. Shut / no shut controller T1 to clear calls.
Update IOS to avoid future occourings.
08-23-2010 12:20 PM
Remember that shut/no shut will remove ALL calls, not just the ones that are hung!!!
08-23-2010 03:01 PM
(C3845-SPSERVICESK9-M), Version 12.4(22)T. Actually the calls will not hang there forever. I noticed that they usually only hang there for less than one minute. I just shut and no shut the the controller where there is no call active.
08-23-2010 03:42 PM
IOS used is not of the most stable, however it could be a telco problem where they fail to send disconnect in some cases.
You can only troubleshoot loggin debug isdn q931 to buffer or server.
08-24-2010 07:03 AM
(C3845-SPSERVICESK9-M), Version 12.4(22)T. Actually the calls will not hang there forever. I noticed that they usually only hang there for less than one minute.
That sounds like the media inactivity timer may be popping. Do you have that configured under the 'gateway' sub-configuration? Ideally, the call should have a disconnect coming in from one of the sides.
If you can get some debugs during one of these scenarios, we can take a look. If this is h323, you can also get 'sh h323 gateway cause' and see which value increments when the call drops.
debug voip ccapi inout
debug isdn q931 (assuming this is a PRI. Use 'debug vpm sig' if it isn't a PRI)
debug h225 asn1 (if this is h323)
debug ccsip mess (if this is SIP)
Collect debugs as follows.
Router(config)# service sequence
Router(config)# service timestamps debug datetime msec
Router(config)# logging buffered 10000000 7
Router(config)# no logging con
Router(config)# no logging mon
Router(config)# voice iec syslog
Router# term len 0
Router# sh logg
Note ANI/DNIS of a call with the issue.
08-24-2010 08:27 AM
Thanks a lot for the detailed step. I will try them today and let you guys know.
Thanks.
Lou
09-10-2010 08:54 AM
Finally I get some time to do more troubleshooting today. I enabled the debug as you suggested.
debug isdn error is ON.
debug isdn q931 is ON. (filter is OFF)
H.225:
H.225 ASN1 Messages debugging is on
CCAPI:
debug voip ccapi inout is ON (filter is OFF)
Keep showing log constantly every 3 seconds. I captured the output. The log repeat a lot in the capture since I show log every 3 seconds. The stuck call number is 3147502946. From the cisco CDR Analysis and Reporting. This call started at Sep 10, 2010 10:05:39 AM and ended at Sep 10, 2010 10:07:22 AM lasting 107 seconds. I checked the log in gateway. The connected time is 10:05:40 AM but disconnected time is 10:20:20 AM lasting 880 seconds.
I check the h323 cause code and didn't see non-normal call clearing number increased. All increased number is normal call clearing. No other suspicious message between connect and disconnect message. The attachement is the capture.
This is really weired. It seems Call Manager already disconnect the call but gateway still connected. Not sure it's AT&T problem or not.
Please help. Thanks a lot!
Lou
09-10-2010 12:51 PM
I took a look at the debug output again. The call process was happening like below:
1. 10:05:39 AM, call get into the T1. call setup. Call get into the queue waiting.
2. 10:07:22 AM. the agent is available. the call was out of queue.
3. 10:08:29 AM. the call ended according to CM CDR and cisco supervisor desktop. See the cc_api_set_called_ccm_detected, cc_api_voice_mode_event, ccCallNotify. cc_api_get_ccm_detected, cc_api_get_ssCTreRoutingNotSupported. no DISCONNECT related message popped up at this time.
4. 10:20:20 AM. The call really disconnected from T1. T1 slot got released.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide