Upgrading IOS remotely on 75 routers...

Answered Question
Sep 21st, 2007

Recently I was asked in my new position to come up with a plan to remotely upgrade the IOS of a large number of routers. I have upgraded a grand total of 2 in my entire life so I am making sure I have a good plan of attach before I start this project. All routers need to be remotely upgraded, and I know (from reading today) about TFTP'g, copying files over, etc, etc. My question first is this, is there such a thing as a PUSH IOS upgrade? Meaning if I do one manually correctly, then another, then another, and am then sure my upgrade plan will work, is there a way to PUSH the new IOS onto all 50 routers (of course all are the same model and currently running the same IOS). This project is still a few weeks away, but thank you for your reponses.

I have this problem too.
0 votes
Correct Answer by Joseph W. Doherty about 9 years 2 months ago

There's really no "wisdom" in doing upgrades manually. Just trade-offs in expected effort. If there were hundreds to upgrade or you're upgrading once a week, or you already have a package that can do it, then automation is likely the better approach. If you only need to do a few once a year, perhaps manual is the more efficient approach. Your number is high enough, that consideration should be given to automation (which is what prompts your question I suspect). If done manually, some thought should be given to the approach to permit it to be done quickly. For the latter, I've already mentioned using FTP and use command sequences to paste in. Something else I do if doing refreshes on different equipment is do them by similar groups so that the images and command sequences are (hopefully) identical.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (4 ratings)
Loading.
Joseph W. Doherty Fri, 09/21/2007 - 15:36

There are products, some in the Cisco portfolio, that can push IOS images and also restart the system. For instance, I've used WLSE to upgrade multiple Cisco AP IOS images in the wee hours and restart them. Such products work best when the systems are "peas in a pod", both for hardware and software.

When there's diversity in the environment, little things such as different sizes for flashes can trip you up. (The above products often try to account for some of this.)

I will often try to figure what will actually take me less time, configuring a product to mass download a IOS image update, or just do them manually.

Often I can run through a manual mass download pretty quickly. I'll open 5 or so telnet sessions, and paste in the command lines to download the image. Using a FTP server (not TFTP) and with a T-3 hub link and remote T-1 links, I can usually upgrade 50 routers in less than an hour. I then schedule the router to reload at a time of my choosing.

Something that can be a problem is defining an effective fall back strategy, especially if the remote router has insufficient flash to hold multiple images. (Another reason I like to manually monitor the upgrade.)

Another possible approach is sending a replacement flash module with the new image. If you're in no rush, you could eventually upgrade all you routers using just one extra module.

uubozou11 Sat, 09/22/2007 - 08:29

I will follow your wisdom and do them manually. Probably making it more complicated than I needed too.

Thank you.

Correct Answer
Joseph W. Doherty Sat, 09/22/2007 - 13:40

There's really no "wisdom" in doing upgrades manually. Just trade-offs in expected effort. If there were hundreds to upgrade or you're upgrading once a week, or you already have a package that can do it, then automation is likely the better approach. If you only need to do a few once a year, perhaps manual is the more efficient approach. Your number is high enough, that consideration should be given to automation (which is what prompts your question I suspect). If done manually, some thought should be given to the approach to permit it to be done quickly. For the latter, I've already mentioned using FTP and use command sequences to paste in. Something else I do if doing refreshes on different equipment is do them by similar groups so that the images and command sequences are (hopefully) identical.

scottmac Sat, 09/22/2007 - 09:31

Cisco's management suite (CiscoWorks) can push new inmages (and a lot more). There are some other third party suites that can also be very helpful.

If you have done any programming in the past, you may want to look at using / dusting off / learning Perl and using tools like Net::Telnet::Cisco to automate the process without purchasing the management packages.

You may want to investigate COSI (Cisco-centric Open Exchange Community):

http://cosi-nms.sourceforge.net/

You may or may not find a tool or two there that could be helpful for this case and in the future.

If you don't have a hard deadline for this (or the deadline is a ways out), then you might want to just set up a schedule to do ~ five a night for two weeks.

Also I'd recommend 3CDaemon for a TFTP server (must be version 2.0 or above, v 2.10 is current, I think) because it doesn't have the 32M limitation that many of the other TFTP servers have.

Some of the bigger images (ISR, Emterprise ...) can be bigger then the 32M limit.

Good Luck

Scott

uubozou11 Sun, 09/23/2007 - 10:49

Thanks for both your replies. I am still weighing the options of both (manual vs. automated) and luckily I was handed this project early enough to where I am still a 2-3 weeks before I even need to start the upgrade (mainly to get PCI compliance on our routers). If I were to go the automated way, can you provide me a name of any program that you have had success with?

Thanks

Danilo Dy Mon, 09/24/2007 - 05:44

Hi,

Since you have a schedule to perform the upgrade (2-3 weeks ahead), that is not plenty of time to introduce new things that even you is not familiar at this time, I recommend the option you have tried before (2x upgrade) which is manually.

I recommend upgrading one-by-one, if possible leave the old IOS in the FLASH. Configure the router to boot to the new IOS (first) and to the old IOS (last), in case there is a problem with your new IOS, the router can still boot to the old IOS. You can start uploading the new IOS now for routers with plenty of FLASH space for two IOS (old and new) and configure the order of boot (new, then old). Schedule one router to reboot on day/night (whatever), once you get more comfortable, you can schedule multiple routers to reboot in one day/night. Take note of the configuration registry :)

After you painstakingly upgrade the routers manually one-by-one, now you can plan for future upgrade and either suggest/justify to your boss to purchase applications to perform automatic upgrade), and of course you will have plenty of time testing that application in a LAB environment.

Regards,

Dandy

uubozou11 Wed, 09/26/2007 - 08:50

I have upgraded some IOS, but never on this scale. But an update, I found out I am more or less going to be assisting one of our senior guys with this and so I will be taking more of a back seat (IOW, learning from him). Thanks to all on your feedback though.

:)

Actions

This Discussion