ā12-16-2009 05:20 AM - edited ā03-06-2019 08:58 AM
Hi,
One of my engineers issued a command to turn off port security on a number of ports using the range command. The command failed on the first attempt due to a tacacs auth failure which I suspect is due to a low tacacs timeout value. The engineer then reduced the number of ports in the range command and re-issued the config change after which the switch just crashed and rebooted.
The logging buffer on the switch displays the following:
000072: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: System previously crashed with the following message:
000073: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 12.2(50)SE3, RELEASE SOFTWARE (fc1)
000074: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Technical Support: http://www.cisco.com/techsupport
000075: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Copyright (c) 1986-2009 by Cisco Systems, Inc.
000076: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Compiled Wed 22-Jul-09 07:03 by prod_rel_team
000077: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000078: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Data TLB Miss Exception (0x1100)!
000079: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000080: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: SRR0 = 0x00B2F2B0 SRR1 = 0x00029230 SRR2 = 0x0064A544 SRR3 = 0x00021200
000081: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: ESR = 0x00000000 DEAR = 0x0D0D1D79 TSR = 0x8C000000 DBSR = 0x00000000
000082: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000083: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: CPU Register Context:
000084: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Vector = 0x00001100 PC = 0x00B2F2B0 MSR = 0x00029230 CR = 0x33005055
000085: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: LR = 0x00B320F0 CTR = 0x0065F470 XER = 0xA0000040
000086: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R0 = 0x00B320F0 R1 = 0x03382450 R2 = 0x00000000 R3 = 0x0345AEE4
000087: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R4 = 0x3C31819D R5 = 0x0000001C R6 = 0x00003C31 R7 = 0x3C31819D
000088: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R8 = 0x00000000 R9 = 0x0D0D0D0D R10 = 0x019F0000 R11 = 0x00000000
000089: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R12 = 0xA0000000 R13 = 0x00110000 R14 = 0x011F406C R15 = 0x00000000
000090: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R16 = 0x0231477C R17 = 0x0231477C R18 = 0x00000006 R19 = 0xFFFFFFFF
000091: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R20 = 0x00000001 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x01D20000
000092: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R24 = 0x00000000 R25 = 0x00000000 R26 = 0x00000000 R27 = 0x01FC9D44
000093: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: R28 = 0x033824C0 R29 = 0x00000000 R30 = 0x0345AEE4 R31 = 0x0345AEE4
000094: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
000095: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Stack trace:
000096: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: PC = 0x00B2F2B0, SP = 0x03382450
000097: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 00: SP = 0x033824A0 PC = 0x59EF102B
000098: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 01: SP = 0x03382500 PC = 0x00B320F0
000099: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 02: SP = 0x03382538 PC = 0x00B310BC
000100: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 03: SP = 0x033825E8 PC = 0x00B3F96C
000101: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 04: SP = 0x03382600 PC = 0x00B40930
000102: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 05: SP = 0x03382610 PC = 0x00B3FEAC
000103: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 06: SP = 0x03382638 PC = 0x00B3078C
000104: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 07: SP = 0x03382648 PC = 0x00B30180
000105: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 08: SP = 0x03382658 PC = 0x00B45358
000106: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 09: SP = 0x033826F0 PC = 0x00B6BE98
000107: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 10: SP = 0x03382718 PC = 0x012022FC
000108: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 11: SP = 0x03382738 PC = 0x011F4334
000109: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 12: SP = 0x03382740 PC = 0x0108EE50
000110: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED: Frame 13: SP = 0x00000000 PC = 0x01085920
000111: *Mar 1 00:03:00 GMT: %PLATFORM-1-CRASHED:
I have done some searching and this could be related to bug CSCsq71492. Has anyone got any idea's where we should go next ?
I have tried using the output interpreter but it is still down.
Chris.
ā12-16-2009 06:41 AM
I have not seen this.
Is there any config in the switch?
Is there an error file in the directory, or a crash file?
This may sound like something to send to TAC for investigation.
ā12-16-2009 06:45 AM
Hi,
Yes there is a config on the switch, we also have a crashinfo file in flash. Unfortunately this switch isn't under a support contract nor is it under smartnet (unless you get a standard level of support for new switches). So we are stuck unless someone here has any recommendations.
ā12-16-2009 06:55 AM
Can you post the crashinfo file?
Regards,
jerry
ā12-16-2009 06:57 AM
Hi
what is the IOS version you are running ? Best thing would be to upgrade the IOS to the latest one...
the bug ID you gave me doesnt correspond to this error... it relates to something as given below:
CSCsq71492 Bug Details
Symptoms: A Cisco IOS device may reload with an address error or have alignment errors and tracebacks such as %ALIGN-3-SPURIOUS or %ALIGN-3-TRACE
Output interpretor shows the following:
%PLATFORM-1-CRASHED: [chars]
Explanation:This message indicate that the system is attempting to display the
failure message from a previous failure. [chars] is the description of the error
message.
Recommended Action: Use the exception crashinfo command to enable 'core dump'
on the switch that crashed. The core dump file will be able to a capture a more
detailed crash and have enough information to determine a root cause. Consider
an IOS upgrade.
Is the switch rebooting continuously ?
Raj
ā12-16-2009 07:08 AM
Crashinfo file attached.
The reason I linked it to bug CSCsq71492 is because we were seeing "command authorisation failures" from tacacs just before the switch crashed, which are usually as a result of the tacacs server timing out (assuming the user has the correct level of authorisation in ACS).
ā12-16-2009 07:11 AM
ā12-16-2009 07:19 AM
ā12-16-2009 07:24 AM
Is this caveat not specific to a stack, presumable the c3750 / c3750-E ?
We saw the crash after issuing port security related commands, not after reloading a switch that has port security enabled.
Thanks for your help btw
ā12-16-2009 07:26 AM
It says 3750, but it could also be with 2960 since the release notes is common between 3750,3560 & 2960 ! bugs can be between platforms.. In any case,
what is the status of the device now ? hanged ? not accessible via console? accessible ?
Raj
ā12-16-2009 07:33 AM
The switch appears to be running perfectly fine now. I think the best thing we could do is upgrade to the latest release which seems to have the least number of open caveats, i.e. 12.2(53)SE
What do you think ?
ā12-16-2009 07:35 AM
Yes.. thats the way to go ! upgrade it and am sure it will solve your issue... unless your switch is hit with some other bug
Raj
ā12-16-2009 02:04 PM
12.2(53)SE??? You mean 12.2(52)SE.
ā12-16-2009 02:15 PM
12.2(52)SE
min flash 32 mb
min dram 64 mb
Hope this helps.. all the best
Raj
ā12-16-2009 08:08 AM
Don't believe this is cause by CSCsx25276. In the crashinfo file, I see the interface range command is involved, and could be related to CSCsz13490. However, I am not able to find the excact match.
Regards,
jerry
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide