×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Catalyst 2960 go crazy with 300++ days uptime

Answered Question
Jul 12th, 2009
User Badges:

We have a bunch of cisco catalyst 2960 switches (WS-C2960G-24TC-L, WS-C2960-48TT-L) with various IOS 12.2(25)SEE3, 12.2(35)SE1 with uptime 300 days and more that have gone mad: they answers very slow by snmp (or just doesn't answer), may not answer from ping, http almost don't work (it may allow you to log in but after that just hangs) and finally cisco don't allow you to login via telnet!!! It just prompts you for username and after you enter first char of it it just closes the session. There are !no errors! in the cisco log and after reboot everything is working ok. What is going on?

Correct Answer by Leo Laohoo about 8 years 1 month ago

Firstly, you can pump the IOS anytime during the day and set a reboot/reload during the night or low traffic. I'd recommend a Change Control.


Next, "guarantee". Ain't no guarantee in place. This is why I am not recommending any versions. Choose a version of your liking and read the Release Notes.


I've been in organizations where people just configure and rack appliances without checking if the firmware running is the standard set by the company. In the end, it's people like you who has to stick their neck out to fix careless implementations like this.


Just to give you an idea what I'm using for my switches. I'm standardizing 2960/3560/3750 IOS to 12.2(46)SE and bootstrap is 12.2(44)SE5. I know you can't get this bootstrap version on the IOS but I upgrade to the newer 12.2(50) IOS for the bootstrap upgrade and downgrade the IOS.


Hope this helps.



  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4 (2 ratings)
Loading.
Leo Laohoo Sun, 07/12/2009 - 22:47
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Sounds like IOS bugs. Don't you want to consider upgrading the IOS? They seem pretty old.

minko_at_work Mon, 07/13/2009 - 00:21
User Badges:

The problem is that we have several dozens of cisco catalysts. And we don't have the permission to perform ios upgrade during worktime. And finally, where is the guarantee that this the IOS problem and the situation doesn't happen again? We have to be sure that this is ios bug and the bug is fixed. Please, help!

iyde Mon, 07/13/2009 - 03:41
User Badges:
  • Silver, 250 points or more

Could be a memory leaking issue. Try checking the "show mem stat" output regularly and see if the free mem and the max mem is decreasing.

HTH

minko_at_work Mon, 07/13/2009 - 04:46
User Badges:

We have checked free mem value and it seems to be just fine:

Processor Pool Total: 42293608 Used: 6922172 Free: 35371436

I/O Pool Total: 4186112 Used: 1654528 Free: 2531584

CPU usage also normal: 12-15%


To observe a long-time changes we need another year of observations and anyway it won't fix IOS problems. I don't believe that there were no such problems with cisco catalyst before.


We just need to make descision now.


Correct Answer
Leo Laohoo Mon, 07/13/2009 - 15:57
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Firstly, you can pump the IOS anytime during the day and set a reboot/reload during the night or low traffic. I'd recommend a Change Control.


Next, "guarantee". Ain't no guarantee in place. This is why I am not recommending any versions. Choose a version of your liking and read the Release Notes.


I've been in organizations where people just configure and rack appliances without checking if the firmware running is the standard set by the company. In the end, it's people like you who has to stick their neck out to fix careless implementations like this.


Just to give you an idea what I'm using for my switches. I'm standardizing 2960/3560/3750 IOS to 12.2(46)SE and bootstrap is 12.2(44)SE5. I know you can't get this bootstrap version on the IOS but I upgrade to the newer 12.2(50) IOS for the bootstrap upgrade and downgrade the IOS.


Hope this helps.



hobbe Tue, 07/14/2009 - 00:18
User Badges:
  • Gold, 750 points or more

ok lets make this an example case.

1) choose an ios you think will work for you.


2) test all your configurations for bugs and faliures on the spare equipment you ofcourse have incase of an emergency.

since you have WS-C2960G-24TC-L and WS-C2960-48TT-L you ofcourse have atleast one of each laying around incase a powersupply breaks down and so on.


3) when you have tested and checked and it works just fine, copy the IOS to the flash of the devices, I personally only use the .BIN file and dont bother with the http version. less code = less bugs and problems. prepare the equipment and make any neccesary changes.


4) anounce that a network outage will occur at day X at time Y "to whom it may cencern" and that this is due to service upgrade in the network. do this some time in advance.


5) stay at workt that day and do the upgrade offworking hours. (set boot system variable and reload the switches)

check to make shure things are coming up and working.


6) Sit down and plan a contingency plan and the next upgrade of the switches.


if an apropriate amount of time passes or a bug arises

go to 1


nobody bothers you when you need to reload the servers after service packs and so on right? or service the car?


Same thing with the network, it also needs some tender care and love.

if it does not get it it will degrade and break down at the worst possible time.


PS. Dont use telnet. use ssh.

There are some nice things you can do with a computer and some usb-multiserial interfaces and a computer that you can always keep in control via the serial interface of the switches.


Good luck.

HTH

Leo Laohoo Tue, 07/14/2009 - 18:46
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

Thanks for the ratings.

Actions

This Discussion