there were 3 in the dir. it did crash twice yesterday.

crashinfo_1

Cisco IOS Software, C3560 Software (C3560-IPBASEK9-M), Version 12.2(50)SE1, RELE
ASE SOFTWARE (fc2)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 06-Apr-09 08:00 by amvarma

Debug Exception (Could be NULL pointer dereference) Exception (0x2000)!

SRR0 = 0x014FF84C  SRR1 = 0x00029230  SRR2 = 0x00694B94  SRR3 = 0x00021000
ESR = 0x00000000  DEAR = 0x00000000  TSR = 0x8C000000  DBSR = 0x10000000

CPU Register Context:
Vector = 0x00002000  PC = 0x00A27A3C  MSR = 0x00029230  CR = 0x33000053
LR = 0x00A27A00  CTR = 0x014F4BB8  XER = 0xE000004A
R0 = 0x00A27A00  R1 = 0x02EE5068  R2 = 0x00000000  R3 = 0x02C16ED4
R4 = 0x00000000  R5 = 0x00000000  R6 = 0x00000000  R7 = 0x00000000
R8 = 0x00007530  R9 = 0x00000000  R10 = 0x00000000  R11 = 0x00000005
R12 = 0xD4F2A233  R13 = 0x00110000  R14 = 0x0169D8E4  R15 = 0x00000000
R16 = 0x00000000  R17 = 0x00000000  R18 = 0x00000000  R19 = 0x00000000
R20 = 0x00000000  R21 = 0x00000000  R22 = 0x00000000  R23 = 0x0000006A
R24 = 0x00000000  R25 = 0x0000006B  R26 = 0x00000000  R27 = 0x02127A10
R28 = 0x03A6A64C  R29 = 0x00000000  R30 = 0x03A6A624  R31 = 0x00000000

Stack trace:
PC = 0x00A27A3C, SP = 0x02EE5068
Frame 00: SP = 0x02EE5078    PC = 0x00A27A00
Frame 01: SP = 0x02EE50B0    PC = 0x014E9AE8
Frame 02: SP = 0x02EE50D0    PC = 0x016D76A4
Frame 03: SP = 0x02EE5108    PC = 0x00C01FF0
Frame 04: SP = 0x02EE5128    PC = 0x0169DBD4
Frame 05: SP = 0x02EE5130    PC = 0x00A299C8
Frame 06: SP = 0x00000000    PC = 0x00A20498

crashinfo_2

Cisco IOS Software, C3560 Software (C3560-IPBASEK9-M), Version 12.2(50)SE1, RELE
ASE SOFTWARE (fc2)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 06-Apr-09 08:00 by amvarma

Debug Exception (Could be NULL pointer dereference) Exception (0x2000)!

SRR0 = 0x014FF84C  SRR1 = 0x00029230  SRR2 = 0x00694B94  SRR3 = 0x00021000
ESR = 0x00000000  DEAR = 0x00000000  TSR = 0x8C000000  DBSR = 0x10000000

CPU Register Context:
Vector = 0x00002000  PC = 0x00A27A3C  MSR = 0x00029230  CR = 0x33005053
LR = 0x00A27A00  CTR = 0x014F4BB8  XER = 0xE000004A
R0 = 0x00A27A00  R1 = 0x02CE9D88  R2 = 0x00000000  R3 = 0x02BE0F74
R4 = 0x00000009  R5 = 0x00000000  R6 = 0x00000000  R7 = 0x00000000
R8 = 0x00007530  R9 = 0x00000000  R10 = 0x00000000  R11 = 0x00000005
R12 = 0xD38290EE  R13 = 0x00110000  R14 = 0x00E92C78  R15 = 0x00000000
R16 = 0x00000000  R17 = 0x00000000  R18 = 0x00000000  R19 = 0x00000000
R20 = 0x00000000  R21 = 0x00000000  R22 = 0x00000000  R23 = 0x00000000
R24 = 0x00000000  R25 = 0x00000000  R26 = 0x00000000  R27 = 0x02127A10
R28 = 0x02B948EC  R29 = 0x00000000  R30 = 0x02B948C4  R31 = 0x00000000

Stack trace:
PC = 0x00A27A3C, SP = 0x02CE9D88
Frame 00: SP = 0x02CE9D98    PC = 0x00A27A00
Frame 01: SP = 0x02CE9DD0    PC = 0x014E9AE8
Frame 02: SP = 0x02CE9DF0    PC = 0x016D76A4
Frame 03: SP = 0x02CE9DF8    PC = 0x00A42D38
Frame 04: SP = 0x02CE9E10    PC = 0x00A420CC
Frame 05: SP = 0x02CE9E28    PC = 0x00A42034
Frame 06: SP = 0x02CE9E38    PC = 0x00A4272C
Frame 07: SP = 0x02CE9E60    PC = 0x0159CE30
Frame 08: SP = 0x02CE9E70    PC = 0x0159D010
Frame 09: SP = 0x02CE9E98    PC = 0x016379C0
Frame 10: SP = 0x02CE9EB8    PC = 0x00E98828
Frame 11: SP = 0x02CE9ED0    PC = 0x00E98BB4
Frame 12: SP = 0x02CE9EE8    PC = 0x00E99000
Frame 13: SP = 0x02CE9EF0    PC = 0x00EA13F4
Frame 14: SP = 0x02CE9F08    PC = 0x00E92B64
Frame 15: SP = 0x02CE9F40    PC = 0x00E93008

crashinfo_3

Switch1#more flash:crashinfo/crashinfo_3
Cisco IOS Software, C3560 Software (C3560-IPBASEK9-M), Version 12.2(50)SE1, RELE
ASE SOFTWARE (fc2)
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Mon 06-Apr-09 08:00 by amvarma

Debug Exception (Could be NULL pointer dereference) Exception (0x2000)!

SRR0 = 0x014FF84C  SRR1 = 0x00029230  SRR2 = 0x00694B94  SRR3 = 0x00021000
ESR = 0x00000000  DEAR = 0x00000000  TSR = 0x8C000000  DBSR = 0x10000000

CPU Register Context:
Vector = 0x00002000  PC = 0x00A27A3C  MSR = 0x00029230  CR = 0x30000053
LR = 0x00A27A00  CTR = 0x014F4BB8  XER = 0xE000004A
R0 = 0x00A27A00  R1 = 0x02B41D88  R2 = 0x00000000  R3 = 0x03A66AF4
R4 = 0x00000009  R5 = 0x00000000  R6 = 0x00000000  R7 = 0x00000000
R8 = 0x00007530  R9 = 0x00000000  R10 = 0x00000000  R11 = 0x00000005
R12 = 0xF321A0F0  R13 = 0x00110000  R14 = 0x00B3A434  R15 = 0x00000000
R16 = 0x00000000  R17 = 0x00000000  R18 = 0x00000000  R19 = 0x00000000
R20 = 0x00000000  R21 = 0x0263C2A8  R22 = 0x079F8E08  R23 = 0x079F8EF9
R24 = 0x029B0450  R25 = 0x00000000  R26 = 0x079F8E04  R27 = 0x02127A10
R28 = 0x026EBED8  R29 = 0x00000000  R30 = 0x026EBEB0  R31 = 0x00000000

Stack trace:
PC = 0x00A27A3C, SP = 0x02B41D88
Frame 00: SP = 0x02B41D98    PC = 0x00A27A00
Frame 01: SP = 0x02B41DD0    PC = 0x014E9AE8
Frame 02: SP = 0x02B41DF0    PC = 0x016D76A4
Frame 03: SP = 0x02B41E18    PC = 0x00B41FF0
Frame 04: SP = 0x02B41EB8    PC = 0x00B41618
Frame 05: SP = 0x02B41EF8    PC = 0x00B3B744
Frame 06: SP = 0x02B41F38    PC = 0x00B3AB34
Frame 07: SP = 0x02B41F40    PC = 0x00A299C8
Frame 08: SP = 0x00000000    PC = 0x00A20498

Looks like an IOS-caused crash.  Upgrade the IOS to 12.2.50-SE4.

> Re: c3560g-24ts random  crash

I'm currently using 12.2(53)SE2 all across the fleet.

Have you checked out .... imagine you have, but just in case.

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807ccc79.shtml#deref

Debug Exception (Could be NULL pointer dereference)

A Catalyst 3560 Series Switch that runs Cisco IOS system software reloads with the

Debug Exception

(Could be NULL pointer dereference)

error message in the logs.

The probable causes for the error message are:

Memory leak in CEF background process. For information on how to resolve this issue, see

Unexpected memory consumption in CEF IPC Background process.

·

overcurrent class. This issue

is documented in Cisco bug ID CSCsa72400 (

registered customers only) .

Also...

https://cisco-support.hosted.jivesoftware.com/docs/DOC-4236?decorator=print

Good luck...

Thanks for the thoughts.. I have checked those before..

> Have you checked out .... imagine you have, but just in case.

> http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807ccc79.shtml#deref

I read that as (a) only affecting 3750s (although some documents mention 3560 too), (b) being related to PoE and this is a 24TS, not 24PS, and (c) I believe some documents mention it was fixed in 12.2(25)SEA and the oldest IOS we sae this in was 12.2(35)SE ..(I think just SE).. but certainly not 12.2(25)SEA.. although bug toolkit still suggests "Release-Pending".

Also some documentation suggests it happens on 3750s when (a) switches are stacked, (b) IP routing is disabled and (c) when the master switch it changed; Since it's 3560 it's not stacked, and IP routing is enabled so they seem to be contraindicated.

SL

thanks for the inputs guys. I looked over that link but from what i saw it did not apply to my 3560's. They dont do POE and the other mentions of 3560 dont really apply either. (though i did setup ntp for them now due to what i read on there about the times.)

there was no console/config connections to the switches that day either as only 2 of us do that here and i know neither of us were on teh switches.

it has not done anything since that day and has been stable so far (knock on wood)

i will look at updating it to a newer ios but where it is a prod switch i just can't update it anytime i want. i have a spare in place and will switch it out if it does it again and continue searching and monitoring it in the meantime.

jimmy i gave u some stars as i did find that time issue in your link and fixed that

Thank you, glad to of helped....I learn alot that way.

FYI TAC have suggested a bug to me which I have replicated twice causing a crash, so I'm confident that's causing it in my case.

If you're interested it's CSCth30052.

SL

thnx SL. I dont use IPv6 at all and was not playing with those settings. Not sure if it would crash it if I did but not going to attempt it right now.