Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Community Member

Lost uplink on 4510E with 6E Sup.

Had a situation on the 18th of January. A Cat 4510E lost connectivity to the network. Obviously my monitoring alerted me to the status and I proceeded to the IDF that the 4510E was located, I was expecting a UPS failure or power issue. On the way I noticed that AP's that were powered off that Switch were still up which told me the swich was still providing power. Infact, once there I saw all the blades were still connected to the end user devices. The only visual anomonaly was the 10 GB X2's showed down on the Sup 6E. These are my uplinks to the 6509 at the core, running Port Channel. At this point all the syslogs showed no events preceding this outage. The only events were the obvious loss of EIGRP adjacency and interface down and line protocol at the 6509. I was not able to get into the switch via the console port as I was hoping something would have been in the local logginng buffer that would have shed some light. I kept getting authentication failure on the console and to do a password recovery would have required a reload anyhow so the switch was reloaded to see if it cleared the condition. Everything came back up including the uplinks. This switch had 2 years of uptime to the day when this ocurred. The IOS version is 12.2(53) SG1. Has anyone experienced this scenario as well?

6 REPLIES
Hall of Fame Super Blue

Lost uplink on 4510E with 6E Sup.

This switch had 2 years of uptime to the day when this ocurred.

Wow.  I never let my switches go >1 year without a reboot.

Community Member

Lost uplink on 4510E with 6E Sup.

Tell me is this considered a best practice? The switch ran since the last IOS upgrade with nary any erros since. No tracebacks or anything that would show memory leaks.In fact this switch is the lowest utilized 4510E on my network. In terms of connections and throughput. Rebooting every year reminds me of the guys that change their oil every 2000 miles in their car or motorcycle. Will it hurt? not really, but is it neccessary? I remember at a previous job we had a 4000 that had 11 years uptime in an old call center location we acquired and had no issues. But we were afraid at that poiint to reload it so that we could upgrade the code. It beame a moot point 3 months later as the call center was closed and the building decommissioned. But that is an exception rather than a rule.

Community Member

Lost uplink on 4510E with 6E Sup.

The Sup module showed a green status light. I realize its just an idioit light but of there was a real issue then I figured it would be amber or red. I am leaning towards and IOS bug but I find no entries when looking it up in cisco's bug tool. I am ready just to write it off as a one onff situation and perhaps it is.

Hall of Fame Super Blue

Lost uplink on 4510E with 6E Sup.

Tell me is this considered a best practice? The switch ran since the last IOS upgrade with nary any erros since. No tracebacks or anything that would show memory leaks.In fact this switch is the lowest utilized 4510E on my network. In terms of connections and throughput. Rebooting every year reminds me of the guys that change their oil every 2000 miles in their car or motorcycle. Will it hurt? not really, but is it neccessary? I remember at a previous job we had a 4000 that had 11 years uptime in an old call center location we acquired and had no issues. But we were afraid at that poiint to reload it so that we could upgrade the code. It beame a moot point 3 months later as the call center was closed and the building decommissioned. But that is an exception rather than a rule.

This is a highly debatable subject.  There are some senior guys/gals who follow of the old adage "if it ain't broke, don't fix it".   Me?  I reboot all my routers, switches, wireless controllers twice a year (or more).  It's all up to you.

Bronze

Lost uplink on 4510E with 6E Sup.

jcroft@scripps.edu wrote:

The only visual anomonaly was the 10 GB X2's showed down on the Sup 6E...I kept getting authentication failure on the console

So the Sup 6E on the 4510E was responsive, but both its 10GB interfaces were marked down.  Rebooting fixed it.

Are you running UDLD?  If so, a stall may have caused it to kick in and disable the Links.  Or it could simply be a software bug.  Or maybe someone hacked in and shut them down manually.  Without logs, it's really impossible to tell. 

A few suggestions

- Have all switches log to a syslog server

- Limit buffered and console logging to the important things (level 5 or lower)

- Have a local username / password setup in your AAA config and memorize it, so you have a quick way in if RADIUS / Tacacs is unavailable

Also wouldn't hurt to upgrade the IOS.  12.2(53)G1 is over 2 years old. 

Community Member

Lost uplink on 4510E with 6E Sup.

Johnny

Thanks for the feedback. as for syslog its been in place, I think loss of uplink killed the chance of it being recoreded, I have buffered logging set to informational  level 6 so thats covered.The local password wouldn't let me in for some reason and given the timeframe  and length of the outage I had to do something to restore. I think your assessment on UDLD is spot on. It makes sense. I only wish I had the buffered logs to verify it. I actually had it scheduled to upgrade the IOS in teh next two weeks after another couple of priority ASA upgrades. As for hacking, my ACS logs show no events on that switch for the previous 9 days and that was a user port activation.

610
Views
0
Helpful
6
Replies
CreatePlease to create content