Impact of enabling UCS L2 disjoint networks for existing pod

[paul.3.robertson]

Servers within our existing v2.0(3c) UCS pod  need new connectivity to a layer 2 network in our datacenter. This network is not accessible to the Juniper switches northbound of our current 6248 fabric interconnects.

After reading the docs, it appears that "disjoint layer 2 networking" may be the answer. If I understand correctly,  we'd cable redundant connections from unused 6248 ports to the switches that host this layer 2 network, and then follow the guide to associate that L2 vlan with those ports.

However, I also understand that I need to tweak the configs for our existing Fabric A and Fabric B port channels that hosts all the "normal" vlans. I'm concerned that doing so may result in network impact for UCS hosts on these networks, or may require a reboot/reacknowledgement of existing UCS hosts. Am I being overly pessimistic?

Cheers,

Paul

[Robert Burns · Accepted solution]

Greetings Paul,

I would indeed schedule an outage window because as you configure the VLAN manager to associate certain VLANs to specific uplinks on each FI, it will re-pinn all vEths which will cause a slight disruption in traffic.  Depending how sensitive your applications are they may or may not be affected - I would error on the side of caution and take a 1 hour maint. window to configure and test it accordingly.

By default, all vlans are allowed on all uplinks.  As soon as you add a VLAN to an uplink/port channel specifically through the VLAN manager, it gets explicitly pruned off all other uplinks/port channels for that FI. 

A great way to check your configuration is after you've configured everything as you see fit, connect to the UCS CLI, "connect nxos" and run "show int trunk".  You should only see the VLANs on the required uplinks - with the exception of VLAN 1 which will be allowed on all uplinks.

Regards,

Robert

[paul.3.robertson]

Robert,

Ouch! I was hoping to avoid any impact at all to existing UCS hosts. :-(

Currently, all northbound traffic flows via a single port-channel on each fabric. If my first step in creating disjoint L2 networks is to configure VLAN manager to associate the existing VLANS with that port group, why would that result in re-pinning? i.e. it's effectively the same traffic flow as before, no? Are you saying that UCS repins regardless? If so, how much is a "slight disruption"?

The following step would then be to add a second disjoint uplink/portchannel with the new VLAN(s), which I assume would cause no impact to the traffic in the (now disjoint) L2 network created in the first step?

[paul.3.robertson]

We imlemented disjoint L2 networks today, and there was no noticeable impact associated with the change. From what I could observe via multiple ping sessions, not a single packet was dropped.

Robert's suggestion to err on the side of caution and schedule a maintenance window is correct. Perhaps we just got lucky today.