I have a 3750 stack of 2 switches. Switch-2 has now rebooted twice in the past week without any error or any apparent reason. No power issues, enviroment, memory and processor indicate normal levels so I don't think any certain process killed the thing.
Any ideas? All commands to try or suggestions of issues are welcome.
My experience leads me to suspect a power issue or hardware failure. In the show version command you should see a line like this:
System returned to ROM by power cycle at 09:37:31 UTC Tue Jan 8 2008 (SP by power on)
If this says you had a reboot due to a power cycle, check your power connections to make sure they are secure. If it says anything else, I would recommend doing a show tech and sending it to TAC. You may have more problems than can be solved here.
This stack has had issues and rebooted once back in the fall so it does have an error like that but its from the last reboot. clear as mud right? The cmd is below, with the unrelated parts deleted out
Compiled Mon 22-May-06 08:52 by yenanh
Image text-base: 0x00003000, data-base: 0x01255B58
ROM: Bootstrap program is C3750 boot loader
BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(25r)SE1, RELEASE SOFTWAR
Roch1-AC-SW06 uptime is 24 weeks, 32 minutes
System returned to ROM by error - Debug Exception (Could be NULL pointer derefer
ence) Exception (0x2000)!, PC 0x90A5D8
System restarted at 07:13:52 EST Tue Nov 18 2008
System image file is "flash:c3750-advipservicesk9-mz.122-25.SEE1.bin"
cisco WS-C3750G-24TS-1U (PowerPC405) processor (revision C0) with 118784K/12280K bytes of memory.
Processor board ID FOC1003Y1WH
Last reset from power-on
3 Virtual Ethernet interfaces
56 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.
512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address : 00:16:C8:65:A8:00
Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 28 WS-C3750G-24TS-1U 12.2(25)SEE1 C3750-ADVIPSERVICESK
2 28 WS-C3750G-24TS-1U 12.2(25)SEE1 C3750-ADVIPSERVICESK
Switch Uptime : 19 hours, 15 minutes
Base ethernet MAC Address : 00:16:C8:72:83:80
Anyone have anymore ideas? This is so random but when its happening we're taking a hit, so I don't want to waste time replacing things if this is a software issue.
Any wild guesses are welcome ...
My first idea is a memory problem or problem with IOS (or did you add the "null" part.
I found this usefull document: with explanation about output interpreter and how to troubleshoot
I suggest to upgrade to check the link above and to upgrade to 12.2.46-SE(ED)or newer to exlude software as reason.
I didn't add the "null" comments, those appeared after it rebooted itself last fall due to mem alloc issues.
But if you look at that closely, that line is referencing the whole switch stack, not the individual switches, and any reason why those would reboot. Any ideas on how I can get that info? Something like "show ver switch-2" I wish it works but it doesn't any ideas?
Thats it! I have been looking at dir: or dirflash: thinking that it would show me all but the second i looked at flash2 I found some of the older crash file indicating a memory allo issue. And a bunch of other stuff in there. Time to investigate! Thanks for the reminder to look at the individual flash
If you look through the old crashinfo file you should see a message about "Debug Exception (Could be NULL pointer der
eference) Exception". If that is the case then that crashinfo file would be helpful for us to look atto track this problem down
Thanks for the rating.
Made the same mistake the first time I handled one of those too!
Can you post the lastest crashinfo so we can verify?
First off, thanks guys for all the help.
Now the upsetting news, the crash info is all from previous crashes in the past year to due to mem allocation issues, and in one case a null pointer exception.
Like I said before, 'sh ver' seems to only be telling me info about switch-1 not switch-2 so if anyone knows how to look at that it would be helpful.
Otherwise, given the time (I'm right in the middle of cutting over about 120 switches from our old 6509/cat/sup1 to 6509/IOS/720) I'm going to be replacing this switch and working on this off online since I have found out that they are running on a mismatched SDM template, and the software should be upgraded, like previously suggested, and this stack didn't have the files loaded correctly to do the auto update.