%PLATFORM-1-CRASHED:

sonyscaria · ‎06-01-2009

I am seeing the following logs on my cisco WS-C3750-24P .... Any idea what could be the issue.

un 2 04:42:00: %PLATFORM-1-CRASHED: System previously crashed with the following message:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Compiled Fri 20-Jul-07 01:58 by nachen

.Jun 2 04:42:00: %PLATFORM-1-CRASHED:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Data TLB Miss Exception (0x1100)!

.Jun 2 04:42:00: %PLATFORM-1-CRASHED:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: SRR0 = 0x00FC6100 SRR1 = 0x00029230 SRR2 = 0x00648994 SRR3 = 0x00021200

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: ESR = 0x00000000 DEAR = 0xA6905FFC TSR = 0x84000000 DBSR = 0x00000000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: CPU Register Context:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Vector = 0x00001100 PC = 0x00FC6100 MSR = 0x00029230 CR = 0x33555039

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: LR = 0x00FC6644 CTR = 0x0032907C XER = 0x80000050

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R0 = 0x00FC6644 R1 = 0x023445E0 R2 = 0x00000000 R3 = 0x00000000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R4 = 0xA6906000 R5 = 0x00F9A060 R6 = 0x01F07304 R7 = 0x00000004

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R8 = 0x00029230 R9 = 0x01850000 R10 = 0x01C30000 R11 = 0x00000000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R12 = 0x33555039 R13 = 0x00110000 R14 = 0x00000000 R15 = 0x00000000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R16 = 0x01E320A4 R17 = 0x01E3206C R18 = 0x01E30000 R19 = 0x01C30000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R20 = 0x01C30000 R21 = 0x00000000 R22 = 0x01E320A4 R23 = 0x01C30000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R24 = 0x00000004 R25 = 0x00000000 R26 = 0xD0000000 R27 = 0x00F9A060

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: R28 = 0xA6906000 R29 = 0x00000000 R30 = 0xA6906000 R31 = 0x00000000

.Jun 2 04:42:00: %PLATFORM-1-CRASHED:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Stack trace:

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: PC = 0x00FC6100, SP = 0x023445E0

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 00: SP = 0x02344600 PC = 0x01F07304

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 01: SP = 0x02344618 PC = 0x00FC6644

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 02: SP = 0x02344620 PC = 0x00F9A064

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 03: SP = 0x02344630 PC = 0x00329374

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 04: SP = 0x02344640 PC = 0x003291A0

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 05: SP = 0x02344668 PC = 0x003263EC

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 06: SP = 0x02344670 PC = 0x004E2B5C

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 07: SP = 0x02344698 PC = 0x006CE07C

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 08: SP = 0x023446C8 PC = 0x006D1870

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 09: SP = 0x023446E8 PC = 0x006D1C50

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 10: SP = 0x01D4B99C PC = 0x00647E18

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 11: SP = 0x01D4B9AC PC = 0x00647DC0

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 12: SP = 0x01D4B9CC PC = 0x0064A26C

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 13: SP = 0x01D4B9D4 PC = 0x0064A7A4

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 14: SP = 0x01D4BA20 PC = 0x00647EC8

.Jun 2 04:42:00: %PLATFORM-1-CRASHED: Frame 15: SP = 0x01EC82C0 PC = 0x0099D118

.Jun 2 04:42:00: %PLATFORM-1-CRASHED:

Giuseppe Larosa · ‎06-02-2009

Hello SonyS,

if you have enough rights I suggest to use the output interpreter tool.

https://www.cisco.com/pcgi-bin/Support/OutputInterpreter/home.pl

put sh ver and sh stack to get a feedback.

if you haven't access to it you can attach a text file with sh ver and sh stack inside in this thread

Hope to help

Giuseppe

Leo Laohoo · ‎06-02-2009

I think you need to RMA this.

keithsauer507 · ‎04-25-2013

Sorry to revive an old thread, but just exprienced this myself today. Also we have a third party company doing a vulnerability scan on our network. Were not so sure a scan or specially crafted packet from them crashed our switch (3560X) or not.

This happened on the latest IOS on a 3560X 12.2(58)SE2.

This thread here is the only one I see that has a simular crash ouput. We did open a ticket with TAC so we will see if hardware replacment is the answer, or if its a config issue.

Still if it was caused by a pen test, that needs to be resolved! Imagine people with the ability to bring down networks with simple scanning software! That's not hardened at all!

The switch was stuck in a reboot loop so NTP never updated the time, so the crash logs says march 1 even though it was april 25.

Log Buffer (4096 bytes):

*Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Compiled Thu 21-Jul-11 01:34 by prod_rel_team

000033: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED:

000034: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Debug Exception (Could be NULL pointer dereference) Exception (0x2000)!

000035: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED:

000036: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: SRR0 = 0x01A91668 SRR1 = 0x00029230 SRR2 = 0x00F72AC0 SRR3 = 0x00029230

000037: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: ESR = 0x00000000 DEAR = 0x00000000 TSR = 0x84000000 DBSR = 0x01000000

000038: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED:

000039: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: CPU Register Context:

000040: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Vector = 0x00002000 PC = 0x00F72AC0 MSR = 0x00029230 CR = 0x53000035

000041: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: LR = 0x00F6B8F0 CTR = 0x00F87808 XER = 0xA0000045

000042: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R0 = 0x00F6B8F0 R1 = 0x05699678 R2 = 0x00000000 R3 = 0x00000000

000043: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R4 = 0x00000001 R5 = 0x00F9282C R6 = 0x00F9282C R7 = 0x00001020

000044: *Mar 1 00:02:09.394: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R8 = 0x00000001 R9 = 0x03E20000 R10 = 0x00029230 R11 = 0x00000000

000045: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R12 = 0x00020DF4 R13 = 0x0D98279C R14 = 0x00C54CAC R15 = 0x00000000

000046: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x000005B5

000047: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R20 = 0x00000000 R21 = 0x00000000 R22 = 0x056996E8 R23 = 0x000016D0

000048: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R24 = 0xA8B174AA R25 = 0x00000000 R26 = 0x903B44D0 R27 = 0x06239608

000049: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: R28 = 0x05480C30 R29 = 0x00000000 R30 = 0x06239608 R31 = 0x00000000

000050: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED:

000051: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Stack trace:

000052: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: PC = 0x00F72AC0, SP = 0x05699678

000053: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 00: SP = 0x05699680 PC = 0x00000000

000054: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 01: SP = 0x05699690 PC = 0x00F6B8F0

000055: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 02: SP = 0x056996A8 PC = 0x00F905BC

000056: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 03: SP = 0x056996D0 PC = 0x00F85BF8

000057: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 04: SP = 0x056997C8 PC = 0x00F87E84

000058: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 05: SP = 0x05699808 PC = 0x00C6BC40

000059: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 06: SP = 0x05699820 PC = 0x00C91CF8

000060: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 07: SP = 0x05699828 PC = 0x00C39310

000061: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 08: SP = 0x056998A0 PC = 0x00C56A58

000062: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 09: SP = 0x05699918 PC = 0x00C549F4

000063: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 10: SP = 0x05699940 PC = 0x00C54B34

000064: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 11: SP = 0x05699958 PC = 0x00C54D84

000065: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 12: SP = 0x05699960 PC = 0x026FB208

000066: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED: Frame 13: SP = 0x00000000 PC = 0x026F507C

000067: *Mar 1 00:02:09.402: %PLATFORM-CLUSTER_MEMBER_2-1-CRASHED:

keithsauer507 · ‎04-25-2013

Cisco TAC is replacing the switch.

Leo Laohoo · ‎04-25-2013

This happened on the latest IOS on a 3560X  12.2(58)SE2.

I know what TAC has said ... But I have but one stupid question: Have you tried DOWNGRADING the IOS?

NOTE: I hate using the word "downgrade" but your IOS, the entire 12.2(58)SE train, is one of the buggiest IOS I've ever used. I wouldn't even recommend 12.2(58)SE, 15.0(1)SE nor 15.0(2)SE to anyone but people I hate. Try downgrading to 12.2(55)SE7.

ANOTHER NOTE: 15.0(2)SE2 is a little bit stable.

keithsauer507 · ‎04-25-2013

No I havent tried that. I looked what versions were out there and see that this version is still the latest one. I figured it must be OK if its still there and theres not a newer one out there.

Well TAC has my logs and everything from the switch. Maybe it will make its way up the chain along with other peoples issues so they can create a new version that fixes these issues.

When the new switch comes in, I'll see what IOS is on it and I won't go all the way up to 12.2(58)SE series yet.

The switch I moved everything to is a 3750G which is running 12.2(55)SE7.