×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

IP Phones Randomly rebooting 7942, 7971, 7941

Unanswered Question
Dec 27th, 2011
User Badges:

I have been getting reports of phone rebooting and after futther analysis they seem to be loosing power as the ports are being shutdown? Anybody seen similar issues?  Wanted to post before I call TAC, any help would be nice.


Enviornment:

IP Phone Models: 7941-42 7971

IP Phone Load 9-1-1SR1S


Switch 3560X, 3560G

Switch IOS: 12.2(58) SE2


interface GigabitEthernet0/6

switchport access vlan 40

switchport mode access

switchport voice vlan 41

srr-queue bandwidth share 1 30 35 5

priority-queue out

mls qos trust device cisco-phone

mls qos trust cos

auto qos voip cisco-phone

spanning-tree portfast

service-policy input AUTOQOS-SRND4-CISCOPHONE-POLICY



SHOW LOG


000735: Dec 27 2011 09:51:31.421 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down

000736: Dec 27 2011 09:51:32.420 PST: %LINK-3-UPDOWN: Interface GigabitEthernet0/6, changed state to down

000737: Dec 27 2011 09:51:34.969 PST: %LINK-3-UPDOWN: Interface GigabitEthernet0/6, changed state to up

000738: Dec 27 2011 09:51:35.976 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up

000739: Dec 27 2011 09:51:35.976 PST: %SWITCH_QOS_TB-5-TRUST_DEVICE_LOST: cisco-phone no longer detected on port Gi0/6, operational port trust state is now untrusted.

000740: Dec 27 2011 09:51:46.855 PST: %SWITCH_QOS_TB-5-TRUST_DEVICE_DETECTED: cisco-phone detected on port Gi0/6, port's configured trust state is now operational.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 1 (4 ratings)
Loading.
Leo Laohoo Tue, 12/27/2011 - 18:09
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Try running TDR.



1.  Command:  test cable tdr interface ;

2.  Wait for about 5 to 7 seconds for the test to run;

3.  Command:  show cable tdr interface ; and

4.  Post the outputs please.

johnlloyd_13 Tue, 12/27/2011 - 18:35
User Badges:
  • Blue, 1500 points or more

Hey Leo,


Sorry to interrupt this thread. I just would like to know if this command works well for subinterface.


Sent from Cisco Technical Support iPhone App

Leo Laohoo Tue, 12/27/2011 - 19:06
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Hey John,


Won't work on routers.


Subinterfaces?  Can't say I've tried it but depending on your switch, it should.

johnlloyd_13 Tue, 12/27/2011 - 20:22
User Badges:
  • Blue, 1500 points or more

Cool! Thanks! I've plan to test it on one of our 6500s but was afraid it might disrupt other devices on the same main gig port.


Sent from Cisco Technical Support iPhone App

Leo Laohoo Tue, 12/27/2011 - 21:12
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Cool! Thanks! I've plan to test it on one of our 6500s but was afraid it might disrupt other devices on the same main gig port.

Hey John,


Unfortunately I haven't had the chance (not allowed or BANNED) to test this on a 6500. 


So if you have a Gig interface, the TDR might cause the connection to flap.  I also do not know how long it takes.  The 4500 would take 60 seconds but the 2960, 3560 and 3750 takes 5 to 7 seconds.


Please let me know how it goes so I can update the document.  Let me know your supervisor card and the blades you use please.

johnlloyd_13 Tue, 12/27/2011 - 21:42
User Badges:
  • Blue, 1500 points or more

hey leo,


sure will do. what we're using is a 6509 with specs below. i'm also finding the right timing (some device to go down) to do this small test and will give you a feedback. is there any difference when you use TDR between FE vs GE port?


#sh ver

Cisco Internetwork Operating System Software

IOS (tm) s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M), Version 12.2(18)SXF9, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2007 by cisco Systems, Inc.

Compiled Wed 16-May-07 22:09 by kellythw

Image text-base: 0x40101040, data-base: 0x42DB1650


ROM: System Bootstrap, Version 12.2(17r)S4, RELEASE SOFTWARE (fc1)

BOOTLDR: s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M), Version 12.2(18)SXF9, RELEASE SOFTWARE (fc1)


Switch uptime is 13 weeks, 3 days, 9 hours, 16 minutes

Time since Switch switched to active is 13 weeks, 3 days, 9 hours, 16 minutes

System returned to ROM by reload (SP by reload)

System restarted at 04:19:45 SGT Sun Sep 25 2011

System image file is "sup-bootdisk:/s72033-advipservicesk9_wan-mz.122-18.SXF9.bin"



This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.


A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html


If you require further assistance please contact us by sending email to

[email protected].


cisco WS-C6509 (R7000) processor (revision 3.0) with 983008K/65536K bytes of memory.

Processor board ID TBP06200070

SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from s/w reset

SuperLAT software (copyright 1990 by Meridian Technology Corp).

X.25 software, Version 3.0.0.

Bridging software.

TN3270 Emulation software.

132 Virtual Ethernet/IEEE 802.3 interfaces

96 FastEthernet/IEEE 802.3 interfaces

50 Gigabit Ethernet/IEEE 802.3 interfaces

1917K bytes of non-volatile configuration memory.

8192K bytes of packet buffer memory.


65536K bytes of Flash internal SIMM (Sector size 512K).

Configuration register is 0x2102


#sh mod

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   16  SFM-capable 16 port 1000mb GBIC        WS-X6516A-GBIC     SAL0731HMSV

  3   16  SFM-capable 16 port 1000mb GBIC        WS-X6516A-GBIC     SAL0803SSFR

  4   16  SFM-capable 16 port 1000mb GBIC        WS-X6516A-GBIC     SAL09232VPB

  6    2  Supervisor Engine 720 (Active)         WS-SUP720-3BXL     SAL1012GNQY

  8   48  SFM-capable 48-port 10/100 Mbps RJ45   WS-X6548-RJ-45     SAL09020E8K

  9   48  SFM-capable 48-port 10/100 Mbps RJ45   WS-X6548-RJ-45     SAL0713AZE3


Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  000d.659c.5ac0 to 000d.659c.5acf   1.0   7.2(1)       8.5(0.46)RFW Ok

  3  0008.7dca.83f8 to 0008.7dca.8407   3.0   7.2(1)       8.5(0.46)RFW Ok

  4  0014.69d9.c308 to 0014.69d9.c317   4.2   7.2(1)       8.5(0.46)RFW Ok

  6  0014.a981.ef0c to 0014.a981.ef0f   5.2   8.4(2)       12.2(18)SXF9 Ok

  8  0013.199c.bd38 to 0013.199c.bd67   6.1   6.3(1)       8.5(0.46)RFW Ok

  9  000c.ceb5.37f8 to 000c.ceb5.3827   5.2   6.3(1)       8.5(0.46)RFW Ok


Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  6  Policy Feature Card 3       WS-F6K-PFC3BXL     SAL1012GHF1  1.8    Ok

  6  MSFC3 Daughterboard         WS-SUP720          SAL1012GNXA  2.5    Ok


Mod  Online Diag Status

---- -------------------

  1  Pass

  3  Pass

  4  Pass

  6  Pass

  8  Pass

  9  Pass



Edit - It seems the TDR command for subinterfaces isn't supported.


#test cable-diagnostics tdr int g1/8.?

% Unrecognized command

Leo Laohoo Tue, 12/27/2011 - 22:06
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Edit - It seems the TDR command for subinterfaces isn't supported.


#test cable-diagnostics tdr int g1/8.?

% Unrecognized command

TDR doesn't "understand" sub-interface.  It maybe able to see the physical interface.  Try "test cable tdr int g1/8" instead.

danaledford Tue, 12/27/2011 - 18:44
User Badges:

TDR test last run on: December 27 18:29:09



Interface Speed Local pair Pair length        Remote pair Pair status

--------- ----- ---------- ------------------ ----------- --------------------

Gi0/6     100M  Pair A     N/A                N/A         Not Completed      

                Pair B     N/A                N/A         Not Completed      

                Pair C     N/A                N/A         Not Completed      

                Pair D     N/A                N/A         Not Completed      

crb-irvine-sw2#



I will keep monitoring to post info when completed

Leo Laohoo Tue, 12/27/2011 - 19:07
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Hey Dana,


TDR has not completed.  If you are using a 4500 the test would take 60 seconds.  If you are using 2960 or 3560/3750 then the test would take 5 to 7 seconds.  Can you please run the command "sh cable tdr int Gi0/6" again please?

danaledford Tue, 12/27/2011 - 19:20
User Badges:

Interface Speed Local pair Pair length        Remote pair Pair status

--------- ----- ---------- ------------------ ----------- --------------------

Gi0/6     100M  Pair A     N/A                N/A         Not Completed      

                Pair B     N/A                N/A         Not Completed      

                Pair C     N/A                N/A         Not Completed      

                Pair D     N/A                N/A         Not Completed      

crb-irvine-sw2#

Leo Laohoo Tue, 12/27/2011 - 19:42
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

What the ...?


Same results?   Can you run the commands again with another port in question?


Your switches are running 12.2(58)SE2, am I correct?


If you've got results like that, I have a strange feeling your cables from the switch is causing issues.

danaledford Wed, 12/28/2011 - 19:10
User Badges:

I am still encountering the same issue.  I have escalated to TAC.  They have a TECH dump and I am waiting for a call back. Almost 24hrs with no response after sending TECH dump.

Leo Laohoo Wed, 12/28/2011 - 19:15
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Hi Dana,


Thanks for the update.  Have you tried changing the cable nearest the switch?

Kapil Atrish Thu, 08/16/2012 - 17:26
User Badges:

Hi Dana,


Were you able to get the resolution for your issue? I am getting the same issue in my environment too.


thanks,

Kapil

Leo Laohoo Thu, 08/16/2012 - 18:50
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless


Were you able to get the resolution for your issue? I am getting the same issue in my environment too.

What is the exact model of your Cisco switch?

anthony.king Wed, 10/10/2012 - 06:47
User Badges:

I started having this problem as well.  At first, I thought it was the IOS because the 3560G switch it was happening on had pretty old software.  I upgraded the switch and it seemed to have resolved it but I just had the phones on that switch go down again today.  Interestingly, I also had a different switch do this same thing.


Here is something else that is interesting.  We have had a CME with about 20 phones running for several years on a 3560G-24P with no problems.  We recently upgraded ourselves and our parent company to a BE-6000 (they had a Nortel with TDM phones).  Our CME was running in conjunction with the BE-6000 as we pre-deployed phones.  As soon as we put the new 7945 phones out, that switch started having problems.  Nothing had changed with that switch, our CME or our 7945 phones for years and that switch had absolutely no problems.  The new phones we put out had newer firmware so I'm leaning towards the phone firmware and/or CUCM doing something odd.  Also, the phones that were still working on our CME went down when the new phones went down, so it looks like something system wide on the switch, not localized to one port.

Mick Doran Tue, 11/06/2012 - 12:25
User Badges:

We have the issue of phones resetting randomly in a CMBE6000 system with only 48 phones connected. There are 4 locations, 3 connected via EPL and 1 location via point to point T1.

All locations, including the main site are seeing sporadic phone resetting. All the phones are either 7942 or 7962 with SCCP42.9-1-1SR1S app load.

CCM version is 8.5

Over the past month I have initiated two TAC calls. The following has been done with no resolution to the problem:

  • Increase the 'Station Keepalive Interval' to 300 ms from the default 30 ms
  • Erased all of the QoS settings on every swtich and router and utilized Cisco AutoQoS
  • Checked cabling to the CCM/Phones, etc.
  • Set all the phones to Delayed according to the following:

Geometric TCP

#

The Cisco Unified IP Phone firmware 7.2(1) introduced a Geometric TCP mechanism to permit IP Phones to measure the round-trip delay between the IP Phone and Unified CM, then adapt the keepalive timeout value. This provided a very accurate failover mechanism when the network delay is consistent.

However, if the network delay is inconsistent, this mechanism may cause the IP Phones to inaccurately attempt failover. The Cisco Unified IP Phone firmware 8.4(2) introduces the ability for the Network Administrator to disable this behavior, if necessary, through the Detect Unified CM Connection Failure parameter defined on the IP Phone device configuration. The default value is Normal; this Geometric TCP mechanism can be disabled if the parameter is set to Delayed.

Has anyone resolved this issue?

Kapil Atrish Thu, 11/08/2012 - 06:06
User Badges:

One of my client is facing similar issues. There servers are across the WAN in a DC. In my case, one of the remote site is loosing phones intermittently. Other remote sites are working fine at the same time.


Myself and TAC has spent lots of time troubleshooting this and finally pinpointed bad WAN connection at that site but customer is confident that there are no issues in the WAN.


I've also done similar troubleshooting what you mentioned above except disabling "Geometric TCP" which I've done last night.


If the issue continues, my next steps will be to run wireshark at remote site and at DC. If I see pakcet loss/unexpectedly higher delays, I can go back to customer and prove that the issue lies somewhere in the network and not in voice gears.



Thanks,

Kapil

Mick Doran Thu, 11/08/2012 - 07:16
User Badges:

We have as well.  We had another customer that was indeed having WAN issues and once resolved the phones resetting or bouncing between fallback SRST and Call Manager stopped.


In this case I, Point-to-Point T1, we have tested and worked with the Carrier to eliminate the ciruits as the potential problem.


I too have had several TAC cases and spent many hours on this with no resolution or next steps from TAC.


We are of the same opinion that it will take some wireshark captures to truly see what is going on.  I plan to put a system with wireshark on the same switch as the call manager and do port mirroring to monitor the communication to the Call Manager for a 24-48 hour period. 


If you do the same I would recommend that you set filters on Wireshark to only look for the keep alive/SCCP ports or you will be inundated with a lot data.


I am curious as to what you will find.

Leo Laohoo Fri, 01/17/2014 - 13:56
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Can you create a different thread?

Gurudath KS Fri, 01/17/2014 - 12:07
User Badges:

Hi,


Meet to had the same issue,fortunately I had called the fiber testing guy.


when he tested there was a fiber link issue which was joined,when it was cut.


Surprisingly,in the first test with general characterstics it was ok.but when we tested with specific characterstics,we were able to identify.


Regards,

Gurudath

Actions

This Discussion