Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

7937 registers from CCM to SRST

We have a CUCM 8.6.1 installation with a few 7937 conference stations. Some of them sitting in a branch office, behind a WAN link, these phones are unregistering from CUCM, and go into SRST after a few seconds of bootup. They are using the latest (1.4.4) firmware. I've checked all of the configurations, network settings, network stats, but I haven't found any cause of this behavior. At the device log, I've found the following:

[07-21]10:36:59.296|sccp |3|00|setSocket_SkinnyContext: index=1 hMtx=9c0000 socket=416786c8

[07-21]10:36:59.296|sccp |3|00|runRegistrationManager: socketConnected_RegManagerMsgID cm[1] param2=0

[07-21]10:36:59.297|sccp |3|00|registrationEvent_RegistrationManager: REGISTERED(EVENT_CONNECTED) param1=1 param2=0

[07-21]10:36:59.297|sccp |3|00|registered_RegistrationManager: REG_MANAGER_EVENT_CONNECTED old secondaryCallManager=2 secondaryCallManager=1

[07-21]10:36:59.298|sccp |3|00|runSocketMonitor_SocketMonitor: SocketMonitor1 "tSM1" started

[07-21]10:36:59.298|sccp |3|00|socketMonitorThread: started 1

[07-21]10:38:14.021|sccp |4|00|stationKeepAliveThread: Max keep alives missed

[07-21]10:38:14.021|sccp |4|00|Call Manager 0 Bad!!! (1292)

[07-21]10:38:14.021|sccp |3|00|setCallManagerStatus_StationRegistration: index=0 status=1

[07-21]10:38:14.021|sccp |3|00|stopKeepAlive_StationKeepAlive 0

[07-21]10:38:14.022|sccp |3|00|setCallManagerStatus_StationRegistration: index=0 status=0

[07-21]10:38:14.022|sccp |3|00|stopKeepAlive_StationKeepAlive 0

[07-21]10:38:14.022|sccp |4|00|stopSocketMonitor_SocketMonitor: 0

[07-21]10:38:14.022|sccp |3|00|closeSocket_SkinnyContext: index=0 hMtx=9b0000 socket=41653b08

[07-21]10:38:14.023|sccp |4|00|stopSocketMonitor_SocketMonitor: completed

[07-21]10:38:14.023|sccp |3|00|runRegistrationManager: registrationFailed_RegManagerMsgID cm[0] param2=1

[07-21]10:38:14.023|sccp |3|00|registrationEvent_RegistrationManager: REGISTERED(EVENT_REG_FAILED) param1=0 param2=1

The phones registering into SRST reference, and sometimes go back to CUCM for a few minutes or seconds. There are some other skinny phones, but that phones are working correctly.

I've tried modifying the switch configuration with trunk/access mode with and without voice vlan settings, removed the dns and domain name settings from dhcp pool, I've reconfigured the CUCM for working based on IP address instead of DNS name, but the problem is still exists. The other 7937 phones in the HQ and the other branch offices are working correctly, the problem persists only at this site, but I don't know why. I've tried remove a phone from CUCM, and added it again, there was a hard reset on the phones... There is no packet drop or congestion on the WAN link, we have configured proper QoS for the phone's signaling and media, there is no huge traffic on the switches. The ip routing is based on eigrp, there is no flapping, all of the routes are few weeks old.

Everyone's tags (2)
11 REPLIES
New Member

7937 registers from CCM to SRST

I've found the solution... The wan link was ipsec protected, and the phones tried to send oversize packets to CUCM. I've adjusted the TCP MSS size, this setting resolved the issue. I've made some captures with EPC on IOS gateway at the branch side, and found the problematic SCCP packet.

New Member

7937 registers from CCM to SRST

I'm having the same issue with a 7937, but I do not believe I have ipsec protected on...

Can you explain a little further about this and also about the statement " I've adjusted the TCP MSS size"

Where was this setting adjusted? in CUCM? the CUBE device? the switch? the terminating router?

Thanks,

~EK

New Member

7937 registers from CCM to SRST

Hi Ellen,

in may case, there was an ipsec protected link between the phone and the CCM server, and the MTU was lower, than at these two endpoints. When an endpoint starts a TCP conversation with an other (in this case via TCP based Skinny), the originating device sends the TCP MSS value in the TCP SYN packet to the terminating side, and that side based on the own network settings (in this case the MTU) can choose, is it acceptable or not. If the originating MSS size is too high for the terminating side's MTU, that will reply an ICMP unreachable message with the proper, lower TCP MSS value, and the originating side will be use this for the further communication. As I know, it's called PMTUD, as a standard way to avoid IP fragmentation. If you have a bottleneck on the transport path, it can be an IPSEC link or an ADSL connection or whatever between the two endpoints, this negotiated TCP MSS size will be too high for the connection, the oversized packets will be fragmented, but if the DF bit was set, this oversized packet will be lost at the bottleneck. An other source of this issue, if you have a firewall between these endpoints, and block the ICMP messages, this PMTUD cannot going through. The solution for this problem, if you configure the proper TCP MSS value on the transport path somewhere, usually at the bottleneck (on ipsec protected interface or a WAN interface). It's an interface command in Cisco IOS, and has two-way effect, it means, enough to configure it at one point of your transport path, it will work to both direction. When an endpoint (in this case it's the CP-7937) sends a TCP SYN to CCM, this TCP MSS modified interface will check the MSS value in the packet, and if this value is higher than the configured, it will rewrite the MSS in the packet before it forward to CCM, and sends back an ICMP unreachable message to the phone. It's like a man in the middle stuff. For the troubleshooting, you have to check your network path between the phone and the CCM server, and if you have a lower MTU somewhere, you have to tune the TCP MSS value at that bottleneck interface. TCP MSS value means the maximum TCP segment size, what is the layer 4 size of the datagram without headers (MSS = MTU - sizeof(TCPHDR) - sizeof(IPHDR)). You can count the proper size, if you decrease the MTU size with the additional layer 2-3 headers. If you choose a too low value, the communcation may will too resource intensive. On ethernet, the MTU is usually 1500 byte, the TCP MSS value is 1460 byte. For example, if you have a dsl connnection on tha path, there is a 8 byte overhead for the PPPOE header, your MTU will be 1492 byte, and you have to decrease your TCP MSS to 1452 byte. You can check your transport path with an extended ping from your router, with an MSS size datagram setting and DF is set. If your ping will fail, you have to decrease the datagram size of the ping, until it will successful. It will be the value, what you have to set as TCP MSS value for that connection. For example: "ping YOUR_DST_IP size 1460 df-bit".

I hope it helps, you can find information with regards of the PMTUD on CCO:

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml

Regards,

Zsolt

New Member

7937 registers from CCM to SRST

Were you using a GRE tunnel or IPSEC?

In my case I have an IPSEC tunnel and I have the same 7937 problem that you have had.... I had the tunnel in "tunnel mode" so I put both sides of the tunnel in "transport mode" and reset the tunnel and still am having the same issue...

New Member

7937 registers from CCM to SRST

Did you try using adjust TCP MSS?

Try to configure the following on your ipsec protected interface or the LAN interface, what is belonging your voice vlan: "ip tcp adjust-mss 1400".

I've used tunnel-mode ipsec protection, and this command resolved my issue.

New Member

7937 registers from CCM to SRST

I just tried that on the switch voice vlan, the internal router interface, and the external router interface and still the same behavoir.

New Member

Re: 7937 registers from CCM to SRST

So, it's strange... Is there any other phone type, what is working well via this link? Did you try the extended ping from the router closest to phone? If you try this ping command, try it with source address of the voice vlan interface. If the ping doesn't work, try to decrease the size of the datagram until your ping will successful. When I troubleshooted my issue, all other phone types worked well, only the 7937 was bad. I made a packet capture with Embedded Packet Capture on the router, and I saw that, when the phone sent a oversized packet to the CCM, this packet lost. The phone tried resend this packet, but it's never arrived to the CCM... Based on this capture, I've tried this extended ping:

Pecs_3845#ping 192.168.99.33 df-bit size 1455

Type escape sequence to abort.

Sending 5, 1455-byte ICMP Echos to 192.168.99.33, timeout is 2 seconds:

Packet sent with the DF bit set

M.M.M

Success rate is 0 percent (0/5)

Pecs_3845#ping 192.168.99.33 df-bit size 1454

Type escape sequence to abort.

Sending 5, 1454-byte ICMP Echos to 192.168.99.33, timeout is 2 seconds:

Packet sent with the DF bit set

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 8/10/12 ms

Pecs_3845#

I saw, it's tipically a tcp mss issue, I've tuned the mss value, and the problem dissapeared. I've attached both captures to this conversation, you can open it with wireshark. In the bad capture, packet no. 87 is the oversized one, what couldn't go through the link. After the mss config, this packet was only 1400 byte long (as the configured mss size), and the phone worked well.

Regards,

Zsolt

New Member

Re: 7937 registers from CCM to SRST

I must have confused myslef with MTU and TCP MSS....

i had set the MTU to something different and then applied the suggested TCP MSS setting and was still getting the problem...

I went in and put all the MTU back to the default settings (ip mtu 1500) on all interfaces on the router and swtich..

I then went into the voice vlan on the local switch and applied the "ip tcp adjust-mss 1400"

And now everything works...

YOU ARE AWESOME! I've spent the past few days with Cisco TAC looking at this and we never could get anything going until I ran across your post!

Thanks again!

~EK

New Member

Re: 7937 registers from CCM to SRST

Hi Ellen,

you're welcome

So, the big difference is between the MTU and TCP MSS is, the MTU is the biggest L2 frame, what the media able to carry, but the TCP MSS is the maximum size of the L4 datagram without L4 and lower headers (L4 payload). If you tune the MTU, you must set it on all connected endpoints to work well, it's not fun. MTU is always depends on media only, what are you using, but the TCP MSS decreased, when you tweak the L3 path (for example with IPSEC protection). The endpoints choose the right TCP MSS based on local MTU, and doesn't know, if the traffic path contains a bottleneck somewhere. In my opinion the best way is, if this mss tweaking is applied on the ipsec protected interface, because it will help for all of the TCP sessions passing through that interface. This issue can kill any other tcp sessions, for example a PC traffic, if that isn't support fragmenting properly.

Regards,

Zsolt

New Member

Re: 7937 registers from CCM to SRST

Have you noticed any issues with certain model phones dropping packets? I have another issue similar to this.... I have some 8945s and none of the 8945s will register (via SIP). From inside the network, on the router, I do a ping to one of the addresses of the 8945s with the df-bit size 1500 and it will always miss one ping... If i drop the size down to 1000, I do not miss any. If i go to size 1100, same thing, I miss one ping.

I did this same test with the a different model phone in that office and i do not miss any pings at all, no matter what size I put.  This is all local, no IPSEC tunnels, etc. This is just router connected to switch, and phones connected to switch.  I tried putting the ip tcp mss-adjust on the lan interface of the router and it makes no difference.

Do you think this may be the same type of issue as we had with the 7937's or do you think it is something different?

Here is the pinging results that I spoke about:

>> Pinging an 8945 model phone (that will not register via SIP)

ROUTER#ping 10.160.25.102 df-bit size 1000

Type escape sequence to abort.

Sending 5, 1000-byte ICMP Echos to 10.160.25.102, timeout is 2 seconds:

Packet sent with the DF bit set

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

ROUTER#ping 10.160.25.112 df-bit size 1100

Type escape sequence to abort.

Sending 5, 1100-byte ICMP Echos to 10.160.25.112, timeout is 2 seconds:

Packet sent with the DF bit set

!!!!.

Success rate is 80 percent (4/5), round-trip min/avg/max = 1/1/4 ms

>> Pinging a 7945 model phone (that is registered via SIP)

ROUTER#ping 10.160.25.102 df-bit size 1500

Type escape sequence to abort.

Sending 5, 1500-byte ICMP Echos to 10.160.25.102, timeout is 2 seconds:

Packet sent with the DF bit set

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

New Member

Re: 7937 registers from CCM to SRST

Hi Ellen,

unfortunately I have no any experience about this issue. The best way, if you make a packet capture on a monitor switch port, and analyse that, what is the point, where the registration fails. Based on your fw version, you can check the release notes and bug tookit. On native Cisco phones, may be better, if you use SCCP fw instead of SIP version. In my opinion, this behavior at ping is normal, handling of ICMP packets isn't the most important job of these devices.

Regards,

Zsolt

ps: from the release notes: http://www.cisco.com/en/US/docs/voice_ip_comm/cuipph/8941_8945/firmware/9_3_4/english/release_notes/P415_BK_R20BE6A8_00_rn-9_3_4-8941-8945_chapter_00.html#P415_TP_IE361BDF_00

Note

The Cisco Unified IP Phone 8941 and 8945 added the “secure by default” feature in Release 9.3(1). Before phones are upgraded to Firmware Release 9.3(4), the Cisco Unified Communications Manager must have Device Pack 8.0.3(24049)/8.5.1(14070)/8.6.2(22030) or higher installed. The phones require this device pack. If the correct device pack is not installed first, the phones cannot successfully register to the Cisco Unified Communications Manager.



Note


SIP 9.3(4) can be upgraded only from Release 9.3(1) and later. This limitation applies to SIP 9.3(4) only; SCCP 9.3(4) is not affected.

1514
Views
35
Helpful
11
Replies
CreatePlease login to create content