What is it requesting to reload when you see this: %SYS-5-RELOAD:

Unanswered Question
Dec 3rd, 2009

Does anybody know...

When the master in a 3750 stack has a memory allocation error and is about to shut down it issues the command %SYS-5-RELOAD: Reload requested Reload Reason: snmp shutdown request. Is this requesting the reboot of the switch or the stack?

Clearly two different requests, and different impacts. I just had this happen here and we all took the bet that the member switch would take over. Well it didn't and I am attempting to figure out if this is by design or our configuration.

Thanks

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Giuseppe Larosa Thu, 12/03/2009 - 11:55

Hello,

to evaluate your configuration you should attach it.

two useful links

config guide chapter

http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_44_se/configuration/guide/swstack.html#wp1189785

troubleshooting stacks

http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/troubleshooting/switch_stacks.html#wp39415

I may be wrong but I remember that if you manually reload the master all the switches are reloaded.

this should be explained in the upgrade procedure and I've seen it when we did a sw upgrade.

What should happen when reload is caused by a crash or watchdog or other causes has to be investigated

Hope to help

Giuseppe

Lsimancek_2 Fri, 12/04/2009 - 12:54

In my instance the reload command was issued by the watchdog. I'm just curious if this is how it would always respond if the memory was too low to even use the console.

My original thought was the 3570s are similar to chassis system and that a supervisor can reload and the modules will keep switching traffic (just not routed while it reboots) so I was hoping that a master 3750 would react the same. We found out the hard way it does not, and I'm curious if its because the watch dog is sending a reboot stack message (the one above) or if it is reloading (and re-electing a master) because the of the memory running so low it lost it mind for lack of a better term, and needed to reboot the stack. Why is this situation any different then when a master looses power, the member switch just takes over right?

Giuseppe Larosa Sun, 12/06/2009 - 05:22

Hello Lsimancek,

watchdog verifies that a process is not using too much cpu time and in case this happens it can even react with a reload if the process time usage is much more then the threshold(s).

if in your case the master switch was in very low memory (you might have opened another thread about this I remember someone describing a master in very low memory condition some days ago) any process trying to get some memory resources can have triggered the watchdog action.

Someone could say that with a stack we get what we have paid for: they are less expensive that a full modular switch with two supervisors.

At first one could think of performance issues but the speed of the ring built with the stackwise cable is good (32 Gbps, 64 Gbps with 3750E).

I don't know the details of how the stack handles the supervisors of individual member switches, but the master may trigger a full stack reload as you have experienced.

Some communications happen on the stackwise cable: until the master sends out its hello frames the other devices trust him and follows its instructions.

When the master is unplugged or powered off other switch members detect from the missing of these messages and this triggers a new master switch election.

However, if this stack is performing only L2 switching I don't see a reason for using all the memory and I would suspect a software defect that causes memory leakage. If so the risk is that it becomes only a question of time, of uptime and the problem may occurr again.

Hope to help

Giuseppe

Lsimancek_2 Mon, 12/07/2009 - 07:55

Hi Giuseppe,

Yes you do get what you pay for with the 3750s. That's for sure.

As far as the watchdog goes, I'm certain that is what caused this stack to reboot, I'm not questioning that. And I also agree it is due to a software leak. But was I was hoping to learn for my own edification and future planning was if someone knew if the reload command the watchdog issues (%Sys-5-reload) is a reload command with a switch number included or if it was reload command for the whole stack?

In my opinion (at least until someone convinces me else wise) I'd prefer the switch with the memory issue to be rebooted not the whole stack. Or even better yet, have the watch dog issue the reload command switch by switch with time in between so that techincally the whole stack doesn't go 'dark' at the same time.

If you happen to know that would be great, so far I have not gotten anywhere with TAC. Its not causeing a problem I'd just like to know.

Thanks,

Lindsay

Giuseppe Larosa Mon, 12/07/2009 - 09:54

Hello Lindsay,

I can only report my personal opinion and I hope some cisco expert can provide more light on this.

I see watchdog as a low level mechanism that monitors cpu usage and I'm not  sure that it is stack aware.

So if I should open a service request on this issue I would clearly ask this: is IOS on stackable switches built in such a way that watchdog is smart enough to detect that it is dealing with a stack and not with a standalone switch?

Only IOS developers can answer to this question.

Hope to help

Giuseppe

Actions

This Discussion

Related Content