Nexus 5548 FCoE and Port Channel

Answered Question
Mar 6th, 2011

I am working on a design where a dual controller NetApp will connected to a pair of Nexus 5548s via FCoE. The NetApp will have 2 dual port FCoE cards per controller. The plan is to port channel 2 per N5K switch and then create a vPC so that all FCoE cards in the same controller are part of a vPC.

The purpose of the vPC is because these FCoE cards will also carry IP storage traffic and CIFS. The FCoE cards will also provide FC storage access to UCS for boot LUNs and eventually FC VMFS LUNs.

When I create the vfc interfaces what do I bind them to?

Each of the 4 FCoE ports on the NetApp will have its own WWPN so I was thinking I would create 4 vfc interfaces and then bind each to the Ethernet port that it is connected to for the WWPN.

I saw some references to binding the vfc to a port channel but I don't see how that would work as there will be 4 vfc, one for each WWPN.

I have this problem too.
0 votes
Correct Answer by Darren Ramsey about 3 years 1 month ago

Jeremy,

Ran across this in a networker's presentation. Topology #2 matches your question.

Correct Answer by vdsudame about 3 years 1 month ago

when the interface is already part of port channel it not allow you to bind the physical interface in the vfc.

This is from my lab switch

24.09.5010B.1(config-if)# int e1/3
24.09.5010B.1(config-if)# channel-group 11 mode active
24.09.5010B.1(config-if)# int vfc3
24.09.5010B.1(config-if)# bind interface e1/3
ERROR: fcoe_mgr: VFC cannot be bound to interface as it is member port of a port-channel (err_id 0x4207003F)

the same is true vice versa. ie if you have a physical interface bundled in a vfc and if you try to create a port channel for it, it wont allow it.

24.09.5010B.1(config-if)# int e1/4
24.09.5010B.1(config-if)# channel-group 11 mode active
command failed: port not compatible [VFC bound to port]

In this case I have e1/4 already binded to vfc4.

So you will need two VPCs with one link port channel on each side and do the binding of port channel to the vfc instead of the physical interface.

hope this clarifies.


Thanks, Vinayak

  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 5 (3 ratings)
roberbur Tue, 03/08/2011 - 20:25

Jeremey,

You have the right idea.  You would bind each of your 4 vfc's to an ethernet interface, not to the port Channel.  The port channeling is only effective on your Etherent traffic, not FC.

So you could still do two separate VPCs for balancing your Ethernet traffic across the two N5Ks, but your FC traffic still gets pinned individually.

Regards,

Robert

vdsudame Wed, 03/09/2011 - 09:59

we will need to bundle each physical interface in one link port channel and you will use that single link port channel to bundle it in a vfc. After this is done you can vpc the port channel on either side of the nexus switch. So yes, we will have 2 vpcs but we will have to bind the vfc in single link port channel and not physical ethernet interface. If you bind it in physical interface, then we cant do vpc for that interface because the vpc config can only go under the port channel interface and not the physical interface.

Jeremy Waldrop Thu, 03/10/2011 - 03:54

Vinayak, I don't think I follow what you are saying. I would like my end result to be a single vPC of 4 FCoE storage ports (2 connected to each 5548).

So assuming my 2 FCoE storage cards are connected to E1/5-6 would the config look like this?

feature fcoe

feature vpc

feature lacp

vsan database

   vsan 11 name fabric a

vlan 811

  fcoe vsan 11

interface Ethernet1/5
  switchport mode trunk
  spanning-tree port type edge trunk
  channel-group 5 mode active
interface Ethernet1/6
  switchport mode trunk
  spanning-tree port type edge trunk
  channel-group 5 mode active

interface port-channel5
  switchport mode trunk
  spanning-tree port type edge trunk
  vpc 5

interface vfc5
  bind interface Ethernet1/5
  no shutdown
interface vfc6
  bind interface Ethernet1/6
  no shutdown

vsan database
  vsan 11 interface vfc5
  vsan 11 interface vfc6

Correct Answer
vdsudame Thu, 03/10/2011 - 09:59

when the interface is already part of port channel it not allow you to bind the physical interface in the vfc.

This is from my lab switch

24.09.5010B.1(config-if)# int e1/3
24.09.5010B.1(config-if)# channel-group 11 mode active
24.09.5010B.1(config-if)# int vfc3
24.09.5010B.1(config-if)# bind interface e1/3
ERROR: fcoe_mgr: VFC cannot be bound to interface as it is member port of a port-channel (err_id 0x4207003F)

the same is true vice versa. ie if you have a physical interface bundled in a vfc and if you try to create a port channel for it, it wont allow it.

24.09.5010B.1(config-if)# int e1/4
24.09.5010B.1(config-if)# channel-group 11 mode active
command failed: port not compatible [VFC bound to port]

In this case I have e1/4 already binded to vfc4.

So you will need two VPCs with one link port channel on each side and do the binding of port channel to the vfc instead of the physical interface.

hope this clarifies.


Thanks, Vinayak

Correct Answer
Darren Ramsey Mon, 03/14/2011 - 17:32

Jeremy,

Ran across this in a networker's presentation. Topology #2 matches your question.

jogentry Tue, 02/19/2013 - 21:13

Darren,

In which session did you find this slide? I've been looking for that forever!

Actions

Login or Register to take actions

This Discussion

Posted March 6, 2011 at 4:07 AM
Stats:
Replies:8 Avg. Rating:5
Views:7818 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard