12-16-2009 05:20 AM - edited 03-06-2019 08:58 AM
Hi,
One of my engineers issued a command to turn off port security on a number of ports using the range command. The command failed on the first attempt due to a tacacs auth failure which I suspect is due to a low tacacs timeout value. The engineer then reduced the number of ports in the range command and re-issued the config change after which the switch just crashed and rebooted.
The logging buffer on the switch displays the following:
000072: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: System previously crashed with the following message:
000073: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 12.2(50)SE3, RELEASE SOFTWARE (fc1)
000074: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Technical Support: http://www.cisco.com/techsupport
000075: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Copyright (c) 1986-2009 by Cisco Systems, Inc.
000076: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Compiled Wed 22-Jul-09 07:03 by prod_rel_team
000077: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000078: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Data TLB Miss Exception (0x1100)!
000079: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000080: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: SRR0 = 0x00B2F2B0 SRR1 = 0x00029230 SRR2 = 0x0064A544 SRR3 = 0x00021200
000081: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: ESR = 0x00000000 DEAR = 0x0D0D1D79 TSR = 0x8C000000 DBSR = 0x00000000
000082: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000083: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: CPU Register Context:
000084: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Vector = 0x00001100 PC = 0x00B2F2B0 MSR = 0x00029230 CR = 0x33005055
000085: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: LR = 0x00B320F0 CTR = 0x0065F470 XER = 0xA0000040
000086: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R0 = 0x00B320F0 R1 = 0x03382450 R2 = 0x00000000 R3 = 0x0345AEE4
000087: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R4 = 0x3C31819D R5 = 0x0000001C R6 = 0x00003C31 R7 = 0x3C31819D
000088: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R8 = 0x00000000 R9 = 0x0D0D0D0D R10 = 0x019F0000 R11 = 0x00000000
000089: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R12 = 0xA0000000 R13 = 0x00110000 R14 = 0x011F406C R15 = 0x00000000
000090: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R16 = 0x0231477C R17 = 0x0231477C R18 = 0x00000006 R19 = 0xFFFFFFFF
000091: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R20 = 0x00000001 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x01D20000
000092: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R24 = 0x00000000 R25 = 0x00000000 R26 = 0x00000000 R27 = 0x01FC9D44
000093: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R28 = 0x033824C0 R29 = 0x00000000 R30 = 0x0345AEE4 R31 = 0x0345AEE4
000094: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000095: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Stack trace:
000096: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: PC = 0x00B2F2B0, SP = 0x03382450
000097: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 00: SP = 0x033824A0 PC = 0x59EF102B
000098: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 01: SP = 0x03382500 PC = 0x00B320F0
000099: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 02: SP = 0x03382538 PC = 0x00B310BC
000100: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 03: SP = 0x033825E8 PC = 0x00B3F96C
000101: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 04: SP = 0x03382600 PC = 0x00B40930
000102: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 05: SP = 0x03382610 PC = 0x00B3FEAC
000103: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 06: SP = 0x03382638 PC = 0x00B3078C
000104: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 07: SP = 0x03382648 PC = 0x00B30180
000105: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 08: SP = 0x03382658 PC = 0x00B45358
000106: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 09: SP = 0x033826F0 PC = 0x00B6BE98
000107: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 10: SP = 0x03382718 PC = 0x012022FC
000108: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 11: SP = 0x03382738 PC = 0x011F4334
000109: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 12: SP = 0x03382740 PC = 0x0108EE50
000110: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 13: SP = 0x00000000 PC = 0x01085920
000111: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
I have done some searching and this could be related to bug CSCsq71492. Has anyone got any idea's where we should go next ?
I have tried using the output interpreter but it is still down.
Chris.
12-16-2009 06:41 AM
I have not seen this.
Is there any config in the switch?
Is there an error file in the directory, or a crash file?
This may sound like something to send to TAC for investigation.
12-16-2009 06:45 AM
Hi,
Yes there is a config on the switch, we also have a crashinfo file in flash. Unfortunately this switch isn't under a support contract nor is it under smartnet (unless you get a standard level of support for new switches). So we are stuck unless someone here has any recommendations.
12-16-2009 06:55 AM
Can you post the crashinfo file?
Regards,
jerry
12-16-2009 06:57 AM
Hi
what is the IOS version you are running ? Best thing would be to upgrade the IOS to the latest one...
the bug ID you gave me doesnt correspond to this error... it relates to something as given below:
CSCsq71492 Bug Details
Symptoms: A Cisco IOS device may reload with an address error or have alignment errors and tracebacks such as %ALIGN-3-SPURIOUS or %ALIGN-3-TRACE
Output interpretor shows the following:
%PLATFORM-1-CRASHED: [chars]
Explanation:This message indicate that the system is attempting to display the
failure message from a previous failure. [chars] is the description of the error
message.
Recommended Action: Use the exception crashinfo command to enable 'core dump'
on the switch that crashed. The core dump file will be able to a capture a more
detailed crash and have enough information to determine a root cause. Consider
an IOS upgrade.
Is the switch rebooting continuously ?
Raj
12-16-2009 07:08 AM
Crashinfo file attached.
The reason I linked it to bug CSCsq71492 is because we were seeing "command authorisation failures" from tacacs just before the switch crashed, which are usually as a result of the tacacs server timing out (assuming the user has the correct level of authorisation in ACS).
12-16-2009 07:11 AM
12-16-2009 07:19 AM
12-16-2009 07:24 AM
Is this caveat not specific to a stack, presumable the c3750 / c3750-E ?
We saw the crash after issuing port security related commands, not after reloading a switch that has port security enabled.
Thanks for your help btw
12-16-2009 07:26 AM
It says 3750, but it could also be with 2960 since the release notes is common between 3750,3560 & 2960 ! bugs can be between platforms.. In any case,
what is the status of the device now ? hanged ? not accessible via console? accessible ?
Raj
12-16-2009 07:33 AM
The switch appears to be running perfectly fine now. I think the best thing we could do is upgrade to the latest release which seems to have the least number of open caveats, i.e. 12.2(53)SE
What do you think ?
12-16-2009 07:35 AM
Yes.. thats the way to go ! upgrade it and am sure it will solve your issue... unless your switch is hit with some other bug
Raj
12-16-2009 02:04 PM
12.2(53)SE??? You mean 12.2(52)SE.
12-16-2009 02:15 PM
12.2(52)SE
min flash 32 mb
min dram 64 mb
Hope this helps.. all the best
Raj
12-16-2009 08:08 AM
Don't believe this is cause by CSCsx25276. In the crashinfo file, I see the interface range command is involved, and could be related to CSCsz13490. However, I am not able to find the excact match.
Regards,
jerry
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: