Nexus 7000 - M132 -> F2 Migration with VPC

james.davis · ‎08-08-2012

So, as the topic indicates we have 2 x N7K's as our serverfarm switches, VPC between them, single vpc, and about 150 vlans.

It was always our intention to upgrade to the F2 line cards when they were made available, rather than the M1 cards that we purchased during the initial upgrade.

The problem we've run into, is there's no way to migrate one chassis at a time, without a complete outage due to the Cross Card VPC limitation. i.e. a F2 card, cannot create a VPC peer to neighbour chassis with a M1 card, even though the port channel comes up.

%VPC-3-VPC_PEER_LINK_BRINGUP_FAILED: vPC peer-link bringup failed (F2 VDC support mismatch)

Creating a new VDC for the F2 cards, and moving networks or creating new ones is not viable as we'll end up with an outage just as long.

The 10-15 minutes of down time required to shutdown both switches to do the upgrade is somewhat unrealistic in our environment, for the next 6 months or so.

Has anyone else run into this issue / migration path, and what was your solution to it.

At this stage, because the port channel comes up, it's likely to be a software limitation to avoid a non supported topology, the easiest solution I can come up with is to ask Cisco for an engineering fix to the code, to allow the non support topology (for a few minutes)

Leo Laohoo · ‎08-08-2012

F2 cards, as far as I'm aware, has to be in it's own VDC. You can't put an F2 card in the same VDC with F1, M1 or M2 card.

james.davis · ‎08-08-2012

Yeah I already noted that, and it was understood before the purchase.

The intention is to replace all of the M1 cards, with F2 cards, one chassis at a time.

i.e. Shutdown chassis B. replace m1 cards with F2, power on, bring up vpc, shutdown chassis A, repeat.

Unfortunately due to the CrossCard VPC issue, i.e F2 in Chassis B, won't bring up a VPC peer link to Chassis A with a M1 card, this can't be done.

The only alternative is to shut down both sides of the server farm switches, pull all of the M1 cards at the same time, and replace with F2.. The result being a total outage which is unacceptable for a HA designed network.

Leo Laohoo · ‎08-08-2012

The only alternative is to shut down both sides of the server farm switches, pull all of the M1 cards at the same time, and replace with F2.. The result being a total outage which is unacceptable for a HA designed network.

How about create a totally different VPC, put the F2 there and start moving things across. Once this is done, switch back to the original VPC.

james.davis · ‎08-08-2012

leolaohoo wrote:

How about create a totally different VPC, put the F2 there and start moving things across. Once this is done, switch back to the original VPC.

I assume you mean create a new VDC with the F2 cards in it, create a new VPC between the switches, and migrate the vlans one by one.

Which unfortunately would require an outage for each of the vlans, rather than one large one, and because we have downstream Nexus 5K's and 2K's with the vlans trunked it wouldn't work.

*edit*

I had already thought of a variation on this:

Create new VDC on each switch

Create new VPC on each switch

Connect original VDC to new VDC at layer 2 - trunk m1 -> f2

Create HSRP entries and spanning tree entries on F2 interfaces

Create uplinks on F2 cards into core.

Add secondary VPC links for 2 sets of downstream 5K's

Shutdown M1 cards and failover to F2 environment

Take the outage hit on the Nexus 2K's that are currently direct attached to the 7K's

A) This is a hack

B) I'm not sure on if the failover would even work

C) This is a hack

D) Loops galore.

E) Time required to implement and cutover extends the risk, fibre capacity etc.

drothan · ‎08-14-2012

James,

We're having the exact same issue. When we coverted our first pair of 7K's we hit this problem when the first F2 Switch was brought up, same F2-M1 Peer-Link incompatibility VPC problem. We were able to avoid an outage because there was only one VPC configured on this pair of 7K's. So what we did was keep the M1 switch running so the servers and NAS filers would stay active, took down the Port-Channel and associated links on the F2 switch for that VPC, moved one of the links out of the Port-Channel and made it a stand-alone trunk, brought that trunk link up, confirmed we had connectivity and then we took down the F2 switch to swap out the cards. On our next pair of 7K's that we need to swap to F2 Modules, there are over 60 VPC's with about 500 VLANs. Obviously there's no way to do the above with that many VPC Port-Channels. Therefore we presented this issue to our Cisco Reps asking them if there is a way to temp turn off the VPC incompatibility check for the F2-M1. No answer yet... I

Leo Laohoo · ‎08-14-2012

Therefore we presented this issue to our Cisco Reps asking them if there is a way to temp turn off the VPC incompatibility check for the F2-M1.  No answer yet...  I

I've asked one of the Cisco Experts about this issue, regarding the F2 being a "princess" and has to be in it's own VDC. I asked whether or not Cisco will develop a software to bridge this gap and all I got was silence.

Word has it is the F2 card has a design flaw in the hardware. So there's a crossroad as to whether develop a software fix or to roll out a new card altogether.

james.davis · ‎08-14-2012

leolaohoo wrote:

Word has it is the F2 card has a design flaw in the hardware. So there's a crossroad as to whether develop a software fix or to roll out a new card altogether.

Not so much a design flaw, but just a completely different forwarding engine, hence the incompatibility.

While I understand the caveat around the F2 in it's own VDC, there still needs to be a migration path from M1 in a HA environment.

I'll be heading to a CPOC in October/November this year to run through some migration "strategies" which will basically be providing a method for the least amount of downtime, currently between myself and the SE's at cisco they best way we can do it is a forced split brain mode, and then a shut / no shut on the physical ports on either side..

jkilleda · ‎03-13-2018

Hi James,

Nexus 7000 - M132 -> F2 Migration with VPC
You can also refer the below document which will help you getting more information on F2 line cards migration with less impact.

https://www.cisco.com/c/en/us/support/docs/switches/nexus-7000-series-switches/116424-technote-vpc-00.html.