Pining Advantage (Pros) and Disadvantage (Cons) in Cisco UCS Fabric Interconnect 6120XP

Unanswered Question
Nov 30th, 2011
User Badges:

Hi,

We have recently installed Cisco UCS devices in our environment with following devices


  1. 2 x Cisco UCS Fabric Interconnect 6120XP with Expansion Module
  2. 2 x Cisco UCS 5108 Chassis
  3. 2 x Cisco UCS 2104XP FEX
  4. 16 x Cisco UCS B 200 M2 Servers with M81KR Mezenine card
  5. 2 x Cisco Catalyst 4948E Switch

Link-Diagram.jpg

On each B200 M2 server we have install ESXi server and have assigned total number of 4 NIC cards which will be used for following network


  1. NIC 1 to carry VM traffic outside world (Public or Private) (This is Pinned to FI-A/2/1)
  2. NIC 2 to carry NetApp 3240 Storage traffic (This is Pinned to FI-A/2/2)
  3. NIC 3 to carry NetApp 3210 Storage traffic(This is Pinned to FI-A/2/2)


We want to know that is there any issue will occurred if we use LAN Pin Groups to use particular uplink to route up strim traffic.


Regards,

Amit Vyas

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Jeremy Waldrop Wed, 11/30/2011 - 13:52
User Badges:
  • Silver, 250 points or more

On the LAN uplinks I would port-channel the links to the 4900 switches. This would be 2 distinct port channels; 1 per Fabric.


With this configuration you would have 1 logical 20g uplink per UCS Fabric Interconnect. With a single logical uplink there isn't a use for Pin Groups.


I personnally am not a fan of using Pin Groups because they add complexity and usually provide inefficient use of the available bandwidth. Having a single 20g logical port-channel provides the best use of bandwidth.


If you didn't port-channel the LAN uplinks and you opted for Pin Groups and the LAN uplink fails the vNICs pinned to that uplink would go down and would not be re-pinned to the other uplink.

Amit Vyas Sat, 12/03/2011 - 06:34
User Badges:

Thank you Jeremy for the reply. We are going with dynamic pinning

chrisb2 Fri, 12/02/2011 - 06:11
User Badges:

That setup will work perfectly, here's the biggest pros/cons of it:


Pros:


1) Separate SAN and LAN traffic

     -Dedicated bandwidth

     -Congestion on one doesn't necessarily mean congestion on the other

2) Additional control of the system


Cons (basically what Jeremy said):


1) No dynamic load-balancing

2) Added Complexity


Things to keep in mind:


-Using this setup if a link fails it will shut down/failover all NICs pinned to it.  If you were using a port-channel (or dynamic pinning) you would have to lose both links for this to happen (if you statically define pinning UCSM won't change it)

.

-10 GB is a lot of traffic - many customers never use even this much and 20GB is more that enough for most deployments

Amit Vyas Sat, 12/03/2011 - 06:31
User Badges:

Thank you Christopher for the reply. We are going with dynamic pinning

kg6itcraig Mon, 12/05/2011 - 09:13
User Badges:

"Dynamic pinning" would be defined as binding a vNIC to a VLAN, and letting the UCS manage traffic flow through ports on FI that "see" that VLAN rather then forcing what path is used by PINing?


Craig

chrisb2 Mon, 12/05/2011 - 09:30
User Badges:

Craig,


That's the idea.


Prior to 2.0 all VLANs were assumed to be on all uplinks (unless in switching mode, but that adds a lot of extra trouble).  Dynamic Pinning in this case is simply allowing the FI to chose which uplinks to use (assuming multiple uplinks, a port-channel appears as one uplink to the code).


2.0 introduced disjoint L2 networks, so now you can specify which VLANs should run on each uplink.  Because of this dynamic pinning now matches the VLANs allowed on a vNIC with the VLANs allowed on a FI uplink. If there isn't an uplink that carries all the VLANs allowed on a vNIC there will be a pinning failure and an error will show up in UCSM and the interface will be down in the OS.

kg6itcraig Mon, 12/05/2011 - 11:45
User Badges:

Thanks!


We are setting up an Isilon NAS POC.


We are in Switch Mode (we suffer from disjointed L2) and running 1.4(3m).


Made a VLAN 2000 and have it created global.


  • There is a 10G switch (n3k) between the FI’s and the Isilon.
  • There is a port-channel from each FI to the 10G switch.
  • VLAN 2000 only exists on that switch in our environment.
  • The vNIC’s are PINNed to those port-channels.
  • We have already POC’ed this works with a Microsoft Storage Server.


Thinking it would be better managed without PINNing? I hear that the UCS can handle it better managing itself, and PINN’in is in general not best practice. 


Craig

Actions

This Discussion