Updating Router/Switch IOS, whats the norm?

[switched switch]

HI All,

Just after some insight... when a device is under maintenance we are able to access any updates in software for that device.

As a network engineer who doesnt have someone above me to ask, does most organisations require that the engineers update the software when they are released? My last co-worker who was senior to me lived very cautiously and didnt ever upgrade the switch or router IOSs.....

Are IOS updates released due to secuirty vulnerabilities, or for bug fixes or both?

Would love to hear what the 'norm' is within the networking community. I am always looking at doing things the 'right' way... it network best practises.

Thanks for anyone who replies...

[Reza Sharifi · Accepted solution]

Hi,

Are IOS updates released due to secuirty vulnerabilities, or for bug fixes or both?

Both

It is a good idea to keep your devices up to date with software as there may be time that you need certain features to use.  That said, it does not mean you upgrade every time there is a new release, as this can cause more headache than help.

I usually upgrade not to the latest but rather to a couple of versions before the latest and greatest.

Also, before any upgrade make sure to read the release note for that version, as there may be risk and vulnerabilities in it that you should know about.

HTH

[Joseph W. Doherty · Accepted solution]

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

If an IOS delivers all the features you need, and there are not (impacting) bugs or security concerns, there's no need to upgrade.  Conversely, if any of the foregoing isn't true, then you'll want to consider an upgrade.

My experience has been, many organizations adopt "if it ain't broke, don't fix it"; but to an extreme.  They will upgrade if there's an impacting (to them) bug and they may upgrade if there's a critical security advisory, but often they don't upgrade when there are new features that might improve their network  operations.

[Leo Laohoo · Accepted solution]

You are not going to get short or succinct answer to this post as this is a very controversial subject. 

There are two schools-of-thoughts:  If-it-ain't-broke-don't-fix-it or upgrade.

I belong to the latter.  I upgrade as much as I can because I find new features.  You have no idea how many times I have to tell other people I work with, "BLAH feature won't work because your switch is running an old IOS."

Another reason why I upgrade is because our security team keeps hassling us with security vulnerabilities.  It's easier for us to upgrade the switches to a known stable IOS than get those annoying notices. 

[switched switch]

Thanks Leo, Joseph and Reza... very much appreciated.

I now have a few more questions:

1) Whats the best way to manage licenses for every end device? I have cisco license manager downloaded but not installed as yet...Is this suitable.

2) How does one safely assume an IOS to be stable prior to upgrade?

3) Can updates be done with minimal downtime for routers and switches?

4) When I have gone to compare features, its a nightmare to work out if all features will be supported in a new image... is there an easy way to validate this besides the feature navigator?

[Leo Laohoo · Accepted solution]

2) How does one safely assume an IOS to be stable prior to upgrade?

There are so many ways to answer this but it all boils down to testing. 

Now I don't have the resources to have a scaled sandpit network environment so what I normally do is load a candidate IOS into a standalone appliance with config and observe for the next 10 business days.  Next, if I don't see any errors or CPU/Memory spike I load this same IOS into two selected appliances somewhere in the network.  If nothing untowards happens in the next 10 business days, I proceed with rolling out the IOS into a site.  Once this is confirm to be working, I roll out to the entire fleet.

3) Can updates be done with minimal downtime for routers and switches?

Depends on your network.  If you don't have any redundancy, no.  You need to reboot the appliance to load the new IOS.  If you have redundancy, there will be impact when the primary goes down and the clients swing across to the secondary.  Now this form of impact will depend entirely upon what kind of clients you have.  I normally schedule my IOS reload at 4am at any day of the week.

4) When I have gone to compare features, its a nightmare to work out if all features will be supported in a new image... is there an easy way to validate this besides the feature navigator?

Stay in the same feature set and you'll be fine.  The most important thing to do is read the Release Notes.  There are many information you can gleam from reading this important document which a lot of people tend to overlook.

1) Whats the best way to manage licenses for every end device? I have cisco license manager downloaded but not installed as yet...Is this suitable.

I've got >900 switches and routers.  Never used the License Manager before and never saw the need.

[switched switch]

Hi Leo,

Thanks for the your insight.. Can i ask then how you best manage 900 devices, all different device models with different IOS'? Surely that must take quite some time unless you have other ways to manage your devices?

Thank you

[Leo Laohoo]

 all different device models with different IOS'?

All same models have one common IOS.  For example, all my 2960/G/S are running 15.0(2)SE4.  All my 3750/G/E/X are running 12.2(55)SE8.

Surely that must take quite some time unless you have other ways to manage your devices?

Monitor using Spectrum but we only enable a few SNMP traps (not all).

[Karsten Iwen]

For example, all my 2960/G/S are running 15.0(2)SE4.  All my 3750/G/E/X are running 12.2(55)SE8.

That leades to the question on which IOS to choose. I deliberately never use versions where the number behind the "SE" is smaller then 4. The smaller the number, the more challenges are included in the IOS.

-- 
Don't stop after you've improved your network! Improve the world by lending money to the working poor:
http://www.kiva.org/invitedby/karsteni

[switched switch]

Yes Karsten, that is exactly that. I dont have sets of stacks available that isnt in production, so how does one go and choose a stable IOS prior to testing.

Is it documented anywhere? Could an email to the TAC get us the answers dependant on what switch models we are upgrading.

[Leo Laohoo]

so how does one go and choose a stable IOS prior to testing.

Pick one and test.  Alternatively, I recommend IOS 12.2(55)SE8 as a first choice and 15.0(2)SE4 as a second choice.

Test either one or both of these IOS.

[switched switch]

Thanks Leo, looks like I may have to rely on others testings (in this case, yours) as it seems it is a 'suck it and see' approach.

[Leo Laohoo]

Thanks Leo, looks like I may have to rely on others testings (in this case, yours) as it seems it is a 'suck it and see' approach.

This forum is great at throwing ideas around. 

In the past few months I've read other people complaining about high CPU and/or high memory utilization of their switches.  When this happens, I would recommend them use either one of the IOS versions I've just posted.  And when some of them do upgrade/downgrade they see remarkable improvements.

Instead of you looking for one, you now have a good version to start testing.