Upgrading 3750 stack in stages

Unanswered Question
May 9th, 2009
User Badges:

Is it possible to upgrade switches in a 3750 stack in phases, instead of doing a reload of the whole stack at once?


If I put the image on the stack master, then do a reload of that switch, will it come back online and automatically start pushing the image out to the other switches one by one and reloading them? Or can i put the image on a few switches, reload them manually and once the come back online do the same for the remaining switches?


We have a stack which dual homes to the different data closets. If we can reload one of the fiber switches at a time, spanning tree will reconverge and keep the down time to a minimum.


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Giuseppe Larosa Sat, 05/09/2009 - 22:14
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Michael,

I don't think you can upgrade on stages only some switches.


see the IOS upgrade procedure for C3750 stacks


http://www.cisco.com/en/US/products/hw/switches/ps5023/products_configuration_example09186a00804799d7.shtml



There are different methods but because the only switch you can access is the master when you reload it all member switches are reloaded


If you try to power cycle only some of them after having loaded the new image you are at the risk of partitioning the stack because of ios version mismatch.


You need a maintanance window.


Further note:

be also aware that if for any reasons a member switch fails to load the new image written on its flash, (it doesn't like it) by the master you need to use the slow Xmodem : rommon does not provide a TFTP option on C3750.

We experienced a nightmare for this reason.


So be careful and ask for an extended time window.


Hope to help

Giuseppe


Joseph W. Doherty Sun, 05/10/2009 - 04:24
User Badges:
  • Super Bronze, 10000 points or more

As Giuseppe notes, a 3750 stack runs one image. It's possible to have different images stored on the indidual switches, but with the risks Giuseppe also notes.


If your downstream devices are dual homed to the stack, and you have an extra port or ports on the upstream side, it might be possible to split the stack into two stacks, each having a link to a downstream device. This then allows you to reload one stack while the other is still carrying production traffic. (Depending on switch configuration and logical topology, it may be possible to reload one switch stack w/o any interruption to production traffic.) BTW, to split the stack, will require some maintenance time.

mhurley131 Sun, 05/10/2009 - 04:32
User Badges:

Yeah, i am probably going to split the stack into two...but then i have to setup HSRP. I was hoping to keep it as one stack so that wouldnt have to be done, but it doesnt look like its going to be possible.


Thanks guys.

volante.canberra Sun, 05/10/2009 - 22:29
User Badges:

You can disable auto-copy-sw

http://cio.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_25_see/command/reference/cli1.html#wp6769756

You can reload just one stack-member:

"This example shows how to reload a specific stack member:


Switch(config)# reload slot 6

Proceed with reload? [confirm]y

"

http://cio.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_25_see/command/reference/cli1.html#wp6776503


iyde Mon, 05/11/2009 - 11:09
User Badges:
  • Silver, 250 points or more

But what happens, then, when you reload the stack master? In my thoughts it will result in at reload of the complete stack.

Anyone who've tried to do this?

mhurley131 Mon, 05/11/2009 - 11:12
User Badges:

You can lose the master of the stack and still have the stack remain operational. That is one of the benefits of having a stack of switches. You can lose any single switch and still have the rest of the switches remain operational.


My concern is having some type of software mismatch and causing the switch stack to segment.

iyde Mon, 05/11/2009 - 11:19
User Badges:
  • Silver, 250 points or more

What I meant was that the previous posting suggested how to reload one switch at a time in order to upgrade it to a newer version. how, then, do we get to upgrade the master switch without the whole stack reloading?

By forcing another switch to be master and then reload the one switch which was previously master?

In other word, I'd like to know whether it's possible to do a swithc-by-switch upgrade without having the downtime, as also asked in the original posting.

Thanks & Regards.

Joseph W. Doherty Mon, 05/11/2009 - 11:45
User Badges:
  • Super Bronze, 10000 points or more

It's not a question of reloading a single switch stack member, but what happens as the 3750s try to function as a stack.


See: http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_50_se/configuration/guide/swstack.html#wp1189553 starting with "All stack members must run the same Cisco IOS software version to ensure compatibility in the stack protocol version among the members. "


[edit]

I suspect, upgrading individual switches within the same train, e.g. 12.2(44)SE1 to 12.2(44)SE2, might be "safe", but jumping versions not as safe.


Even if it's safe, what does it save you? When the master resets, if all other switches are running the "newer" version, you'll may only need to wait for the master to migrate to another stack member. I.e. this might be (is?) faster than full stack reload.


The former master, if it follows the rules for a lost and restarted stack member, likely will not reclaim being a master until there's a new stack election. This has implications such as gateway MAC change. Plus, other services might be interrupted with the master change (some of these might be further mitigated with certain configurations; software and hardware).


The choice then is trying to shave some time off a stack reload to a new version vs. all the issues that might arise attempting to do so incrementally.

iyde Mon, 05/11/2009 - 11:47
User Badges:
  • Silver, 250 points or more

Thanks.

So in other words, there's no feasible way of upgrade a stack in phases in order to take advantage of the NIC teaming from servers to avoid dowbntime for servers?

Joseph W. Doherty Mon, 05/11/2009 - 12:25
User Badges:
  • Super Bronze, 10000 points or more

Considering what Cisco documents, I don't believe bumping individual 3750 stack member switches is a feasible approach for routine operation. It looks like it's possible in some situtations.


If there's a routine requirement for keeping servers on-line, 24/7 365, then the network should be designed with this in mind. The 3750 series, used as a single stack, doesn't, it appears, meet such a requirement.

Giuseppe Larosa Mon, 05/11/2009 - 12:46
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Ingolf, Joseph.


I agree with Joseph and as I wrote in my first post we had a very bad experience in an attempt to upgrade a 4 C3750 stack from 12.2(25)SE to 12.2(44)SE instead of one hour out of service we had extended out of service for hours because two switches didn't like the new image, they have gone in rom monitor and only xmodem worked for them to load an usable ios image.

They were used in a wiring closet and people couldn't work that day.


The release had been suggested by Cisco TAC to try to solve an issue.


So I don't imagine is possible to upgrade one switch of the stack at a time.


Hope to help

Giuseppe


nate-miller Mon, 05/11/2009 - 13:49
User Badges:
  • Bronze, 100 points or more

Correct- if you want your design to include availability during a software upgrade, you'll need two different switches/stacks. Having multiple versions of code running in a stack means you'll end up with version mismatches and be unable to reach boxes without physically removing them from the stack and upgrading them manually. It made for a very long night.


Use the tar files, and use the archive function. It takes a long time, but it's a pretty simple process.


If you're trying to keep uptime at a core level, the best thing to do is split the stack and take the administrative overhead. Granted, the bandwidteh the stack provides is now gone.

Actions

This Discussion