partial reload reason of cisco stack switch

[nagasheshu.2014]

Hi All,

We have cisco stack switch of 3750 with 5 units it as below.

Switch Ports Model              SW Version            SW Image
------ ----- -----              ----------            ----------
     1 52    WS-C3750-48P       12.2(55)SE4           C3750-IPBASEK9-M
     2 52    WS-C3750-48P       12.2(55)SE4           C3750-IPBASEK9-M
     3 52    WS-C3750-48P       12.2(55)SE4           C3750-IPBASEK9-M
    4 52    WS-C3750-48P       12.2(55)SE4           C3750-IPBASEK9-M
     5 54    WS-C3750X-48P      12.2(55)SE4           C3750E-IPBASEK9-M

out of 5, unit 2 and 3 got reloaded on 20/8/2017, and there are no logs in the switch for 20th August.

Customer says there were no power issues in the site and not any local issues.

Is there any way find out the reason why those two units got rebooted with any commands?

Model number                    : WS-C3750-48PS-S
Switch Uptime                   : 1 day, 13 hours, 18 minutes
Model number                    : WS-C3750-48PS-S
Switch Uptime                   : 1 day, 13 hours, 24 minutes

Thanks in advance,

Nagasheshu.

[Leo Laohoo]

Post the complete output to the command "dir flash<SWITCH number>:".  

[nagasheshu.2014]

Hi, please find the details.

Directory of flash2:/

2 -rwx 3496 Mar 1 1993 00:01:54 +00:00 vlan.dat
3 -rwx 1935 Mar 13 2017 19:07:40 +00:00 private-config.text
4 drwx 128 Jun 5 2014 15:23:51 +01:00 crashinfo_ext
5 drwx 192 Mar 26 2012 02:02:07 +01:00 c3750-ipbasek9-mz.122-55.S E4
363 -rwx 49753 Mar 13 2017 19:07:38 +00:00 config.text
454 drwx 6976 Mar 1 1993 00:01:24 +00:00 crashinfo
456 -rwx 41992 Mar 13 2017 19:07:17 +00:00 config.text.backup
458 -rwx 1935 Mar 13 2017 19:07:21 +00:00 private-config.text.backup
498 -rwx 3096 Mar 1 1993 00:03:34 +00:00 multiple-fs

15998976 bytes total (98304 bytes free)

Directory of flash3:/

2 -rwx 3496 Mar 1 1993 00:02:06 +00:00 vlan.dat
3 -rwx 1935 Mar 13 2017 19:07:40 +00:00 private-config.text
4 -rwx 41992 Mar 13 2017 19:07:15 +00:00 config.text.backup
364 drwx 192 Mar 26 2012 02:12:53 +01:00 c3750-ipbasek9-mz.122-55.S E4
363 -rwx 49753 Mar 13 2017 19:07:38 +00:00 config.text
453 drwx 128 Jul 27 2016 12:30:08 +01:00 crashinfo_ext
454 -rwx 1935 Mar 13 2017 19:07:18 +00:00 private-config.text.backup
477 -rwx 3096 Mar 1 1993 00:03:32 +00:00 multiple-fs
457 drwx 4096 Mar 1 1993 00:01:23 +00:00 crashinfo

15998976 bytes total (136192 bytes free)

[GRANT3779]

Hi,

You could try the following "remote command x " - See example from my own stack. I have clipped a lot of the output which is irrelevant. You view specific switch specific info using this command rather than the stack as a whole.

SW-SVR-01#sh switch
Switch/Stack Mac Address : 0021.d750.8d00
H/W Current
Switch# Role Mac Address Priority Version State
----------------------------------------------------------
1 Member 0021.a16d.1100 14 0 Ready
*2 Master 0021.d750.8d00 15 0 Ready
3 Member 9caf.ca2a.e080 13 0 Ready
4 Member 5087.89cf.e080 1 1 Ready

SW-SVR-01#remote command 1 show ver
Switch : 1 :
------------
Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(55)SE11, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Wed 17-Aug-16 13:28 by prod_rel_team
Image text-base: 0x01000000, data-base: 0x02D00000

ROM: Bootstrap program is C3750 boot loader
BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(44)SE5, RELEASE SOFTWARE (fc1)

SW-SVR-01-1 uptime is 6 weeks, 4 days, 23 hours, 3 minutes
System returned to ROM by power-on
System restarted at 17:31:52 TJM Thu Jul 6 2017
System image file is "flash:/c3750-ipbasek9-mz.122-55.SE11.bin"

cisco WS-C3750G-24PS (PowerPC405) processor (revision F0) with 131072K bytes of memory.
Processor board ID FOC1219W5NJ
Last reset from power-on

Unless you were logging to a syslog server and there were some events leading up to the problem you probably won't get much detailed info from the above.

SW-SVR-01#remote command 3 show logg
Switch : 3 :
------------

Syslog logging: enabled (0 messages dropped, 0 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)

No Active Message Discriminator.

No Inactive Message Discriminator.

Console logging: level emergencies, 0 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 108 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Persistent logging: disabled

No active filter modules.

Trap logging: level informational, 73 message lines logged

Log Buffer (4096 bytes):
GRANTED: Interface Fa3/0/39: Power granted
Jul 6 12:53:41: %ILPOWER-5-POWER_GRANTED: granted Interface Fa3/0/40: Power 

[nagasheshu.2014]

Hi,

Thanks for quick response.

Please find the output from device.

Switch : 2 :
------------
Switch-2 uptime is 1 day, 14 hours, 7 minutes
System returned to ROM by address error at PC 0x1BB98D8, address 0x0
System restarted at 23:03:39 BST Sun Aug 20 2017

Switch : 3 :
------------
Switch-3 uptime is 1 day, 14 hours, 13 minutes
System returned to ROM by address error at PC 0x1BB98D8, address 0x0
System restarted at 22:57:51 BST Sun Aug 20 2017

what is "address error at PC 0x1BB98D8, address 0x0"?

Thanks,

Nagasheshu.

[Mark Malone]

Hi No logs even to indicate a switch reloaded thats strange in a stack ?

what does the main show version show under last reset and uptime ?

is there no external syslog setup either ?

is there stack power , redundent PS in the switches or they single PS no stack power

do you have the logs to post , were only going to be guessing what happened without and provided information

[nagasheshu.2014]

Hi,

Switch : 2 :
------------
Switch-2 uptime is 1 day, 14 hours, 7 minutes
System returned to ROM by address error at PC 0x1BB98D8, address 0x0
System restarted at 23:03:39 BST Sun Aug 20 2017

Switch : 3 :
------------
Switch-3 uptime is 1 day, 14 hours, 13 minutes
System returned to ROM by address error at PC 0x1BB98D8, address 0x0
System restarted at 22:57:51 BST Sun Aug 20 2017


Hi No logs even to indicate a switch reloaded thats strange in a stack ? - This was occured on 20Th August, we have logs only from 21st.

what does the main show version show under last reset and uptime ?

sh ver | i Uptime|reset
Last reset from power-on
Switch Uptime                   : 5 weeks, 4 days, 13 hours, 53 minutes
Switch Uptime                   : 1 day, 14 hours, 14 minutes
Switch Uptime                   : 1 day, 14 hours, 19 minutes
Switch Uptime                   : 24 weeks, 2 days, 21 hours, 49 minutes



is there no external syslog setup either ?

Nope.

is there stack power , redundent PS in the switches or they single PS no stack power - Not sure, how to check? Show inventory?

do you have the logs to post , were only going to be guessing what happened without and provided information

As I said, logs have been flushed out.

[GRANT3779]

Personally I would have this logged with TAC and also maybe look at upgrading the IOS. I had a quick look for any bugs which look similar but not found anything that stands out.

[Mark Malone]

thats a bus crash on the switch itself error at PC 0x1BB98D8, address 0x0 , the fact its happened on both switches its most likely software caused it and upgarde should fix it , probably best though as Grant said go through TAC as that way they can give you the bug id and you can be sure the IOS version you pick wont be effected too , TAC will know exactly what the PC error relates too , they dont usually make those public