Solved: Nexus 5548 to Flex Fabric

roger perkin · ‎05-30-2012

I have been tasked with connecting up 2 x Nexus 5548's to an HPc7000 Blade Chassis with a Flex Fabric card, and also to a Netapp 3240

I want to run FCOE and am just looking for info on how to configure this.

Can anyone provide any pointers on where to start looking for this info?

I am going on a Nexus course in a few weeks to get up to speed on the technology, but need to start making some progress on the connectivity.

We already have all the ports populated with SFP's so will be using fibre for connectivity

My experience with Nexus is nil

Thanks

Roger

krun_shah · ‎05-30-2012

A typical Flexfabric with Nexus 5548 setup would looks like this.

Summary steps would be

1. If you have 5548UP (unified port) then you need to choose the range of port which you want to make is as ethernet OR fc port followed by reload of switch.

2. Configure virtual port channel domain, peer link and peer keepalive link between two nexus 5548.

3. Configure port connected to flexfabric module as virtual port-channel.

4. Configure SAN A and SAN B (VSANs) on each switch and configure fc interfaces connected to flex fabric modules as E_ports.

Configuration guide can be found here

http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html

http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02656171/c02656171.pdf

View solution in original post

krun_shah · ‎05-30-2012

A typical Flexfabric with Nexus 5548 setup would looks like this.

Summary steps would be

1. If you have 5548UP (unified port) then you need to choose the range of port which you want to make is as ethernet OR fc port followed by reload of switch.

2. Configure virtual port channel domain, peer link and peer keepalive link between two nexus 5548.

3. Configure port connected to flexfabric module as virtual port-channel.

4. Configure SAN A and SAN B (VSANs) on each switch and configure fc interfaces connected to flex fabric modules as E_ports.

Configuration guide can be found here

http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html

http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c02656171/c02656171.pdf

roger perkin · ‎05-30-2012

So are you saying you cannot do FCOE from the NEXUS to the Flexfabric it is only between the 2 Nexus's?

I will have to wire the ethernet and FC separatley and configure the ports to be either FC or Ethernet

Or is this only if the switch is a UP?

deshongs · ‎05-30-2012

Last I knew this problem was due to the FlexFabric switches, not Nexus. There is no northbound FCoE from the flexfabric switch. It seperates FC and Ethernet and only performs FCoE to the blades within the chassis itself. This certainly might have changed but was still the case last I knew. The Nexus 5548 switches will support multi-hop FCoE using FIP.

One way to support end to end FCoE within the HP C7000 is to purchase the Nexus fabric extenders for that chassis to replace the flexfabric switches.

krun_shah · ‎05-31-2012

I am not sure if Flexfabric can function as lossless ethernet bridge / FIP snooping brige, if it does then you do not need to wire FC cables but instead you use those ports as normal ethernet port and put it in FCoE vlan ONLY. so you do not reduce cabling but all your FCOE traffic traverse thru those dedicated links. This set up will be similar to what nexus 4000 with nexus 5500 as ToR switch does in IBM Bladecenter enviroment.

roger perkin · ‎06-15-2012

I am getting issues connecting Nexus5548 to HP Flexfabric in that the vfc interface keeps errdisabling.

I have 2 x VFC's up to our Netapps, FCOE is working fine.

However trying to do the same to the FlexFabric just errrdisables the port

interface port-channel1

switchport mode trunk

vpc 1

spanning-tree port type edge trunk

interface vfc1

bind interface port-channel1

no shutdown

vfc1 1000 F on errDisabled -- -- --

vfc2 1000 F on errDisabled -- -- --

vfc5 1000 F on trunking -- TF auto --

vfc6 1000 F on trunking -- TF auto --

Has anyone connected a FlexFabric to a Nexus 5548 or will I as suggested above have to buy a Nexus Fabric Extender?

Is this a Nexus 2000 or a dedicated card for the C7000?

thanks

Roger

Carlo Schmidt · ‎06-15-2012

Roger,

Can you include the following:

show platform software fcoe_mgr info interface vfc 1

show platform software fcoe_mgr event-history errors

Thanks,

krun_shah · ‎06-15-2012

Not sure how you connected flexfabric to Nexus 5548!! According to above topology there is no need to have multihop FCoE (VE_port to VE-Port) between flexfabric and Nexus 5548. Flexfabrci decapsulates the FCoE frame and send FC frame to nexus 5548. FCoE only happnes between server CNA and flexfabric not on Nexus 5548.

You just need to configure Fiber channel interface between Nexus 5548 and Flexfabric. Your flexfabric switch can run in NPV mode with Nexus 5548 has NPIV.

If you are using FIP snooping on Flexfabric card then you need to bind the mac address of server insted of port-channel interface.