Nexus 5000 configuration for NetApp 1139A Universal Target Adapters

Unanswered Question
Sep 14th, 2010
User Badges:

I am having problems getting my interface to come up for this configuration. We have installed the UTA (FCoE target cards) in our NetApp heads and cabled them to our Nexus 5Ks. I have the Nexus interfaces configured as follows.


vlan 1010
fcoe vsan 10
name Storage_Mapped_VSAN10


interface e1/15
description NetApp-3140-01-3A
switchport mode trunk
switchport trunk native vlan 1010
switchport trunk allowed vlan 1010
spanning-tree port type edge trunk


interface vfc115
bind interface e1/15
no shut


vsan database
vsan 10 interface vfc115



There is nowhere in the NetApp device to specify the VLAN information for them so I tried doing it as an access port and as a trunk port making the VLAN on the Nexus the native VLAN and only allowed VLAN but it still does not work. It shows up on the NetApp side as Link Not Connected. I don't know if it really matters on the fact that it is running in a NetApp or not so I'll include the QLogic part number I'm dealing with as well.


Fibre Channel Target Adapter 3a (Dual-channel, QLogic CNA 8112 (8152) rev. 2)


Any ideas would be appreciated.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Prashanth Krishnappa Wed, 09/15/2010 - 09:07
User Badges:
  • Cisco Employee,

Configuration should not cause the link to not come up. If the interfaces show up as not connected or down, I would first look into the
physical layer. After this is taken care of, I see you are not allowing your FCoE VLAN on the trunk. You will need to correct this.

kbhagat Thu, 10/07/2010 - 12:21
User Badges:

I'm having the same exact issue -- what am i missing ?

did u get a solution?

JUSTIN LOUCKS Fri, 10/08/2010 - 05:17
User Badges:

No. I have not gotten anywhere yet. I can see the Ethernet side as up in the NetApp but the FC interface is still down in the NetApp and the vFC interface is down in the Nexus.  I got absolutely no where with the NetApp case I opened and gave up. I will probably start looking at it again next week. Let me know if you figure it out between now and then. I think there has to be a privileged command in the NetApp side to configure the FCoE VLAN or something.


You can see the CNA properties under privileged options in the NetApp which shows the Ethernet Link status as up.


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman","serif";}

netapp-3140-01*> cna show


Slot:             3


Description:      Dual-channel, QLogic CNA 8112(8152) rev. 2


FC Adapter Name:  3a


MPI Firmware Rev: 1.38.0


PHY Firmware Rev: 1.6.0


FC Firmware Rev:  5.2.0


FC Port ID:       0x000000


FC Nodename:      50:0a:09:80:89:ca:c7:41 (500a098089cac741)


FC Portname:      50:0a:09:81:99:ca:c7:41 (500a098199cac741)


FC Status:        LINK NOT CONNECTED


Link Speed:       10 GBit


Link Status:      LINK UP




Slot:             3


Description:      Dual-channel, QLogic CNA 8112(8152) rev. 2


FC Adapter Name:  3b


MPI Firmware Rev: 1.38.0


PHY Firmware Rev: 1.6.0


FC Firmware Rev:  5.2.0


FC Port ID:       0x000000


FC Nodename:      50:0a:09:80:89:ca:c7:41 (500a098089cac741)


FC Portname:      50:0a:09:82:99:ca:c7:41 (500a098299cac741)


FC Status:        LINK NOT CONNECTED


Link Speed:       10 GBit


Link Status:      LINK UP

kbhagat Fri, 10/08/2010 - 05:27
User Badges:

I got mine working.

What data ontap are you using?

kbhagat Fri, 10/08/2010 - 10:33
User Badges:

I'll share my configs with u when I get home to night. I'm running ontap 8.01rc1. I had the same issue. All I did was clear the configs and reboot the heads and the links came up


Sent from my iPhone

kbhagat Fri, 10/08/2010 - 20:36
User Badges:

Here are my Configs that worked

I have 2 N5020 with a 3140HA running ontap 8.0.1RC1

The CNA's are  on e1/9 and e1/10 on each n5k switch.. I've only show the onse side but it pretty much the same for the other side only differecen is the vsan #.  I've rebooted the heads a numbe of times with no issues.  I can see the LUNS from MY UCS Blade servers.



---- MY Nexus 5020 -----

NK5-01# sh run
version 4.1(3)N2(1)

feature npiv
feature fcoe

vlan 1010
  fcoe vsan 1000

vsan database
  vsan 1000 name "Fabric-A"

interface vfc9
  bind interface Ethernet1/9
  no shutdown

interface vfc10
  bind interface Ethernet1/10
  no shutdown

vsan database
  vsan 1000 interface vfc9
  vsan 1000 interface vfc10
  vsan 1000 interface fc2/1
  vsan 1000 interface fc2/2

interface fc2/1
  no shutdown

interface fc2/2
  no shutdown


interface Ethernet1/9
  switchport mode trunk
  spanning-tree port type edge trunk

interface Ethernet1/10
  switchport mode trunk
  spanning-tree port type edge trunk

zone name vSphere vsan 1000
    member pwwn 50:0a:09:83:89:ab:8b:e5
    member pwwn 50:0a:09:83:99:ab:8b:e5
    member pwwn 20:00:00:25:b5:00:00:31

zoneset name UCS6100 vsan 1000
    member vSphere

zoneset activate name UCS6100 vsan 1000

NK5-01# sh flogi database
--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME      
--------------------------------------------------------------------------------
fc2/1            1000  0x050003  20:41:00:05:73:a9:8a:c0 23:e8:00:05:73:a9:8a:c1
fc2/1            1000  0x050004  20:00:00:25:b5:00:00:31 20:00:00:25:b5:01:01:8f
fc2/1            1000  0x050006  20:00:00:25:b5:00:00:5f 20:00:00:25:b5:01:01:6f
vfc9             1000  0x050001  50:0a:09:83:99:ab:8b:e5 50:0a:09:80:89:ab:8b:e5
vfc10            1000  0x050002  50:0a:09:83:89:ab:8b:e5 50:0a:09:80:89:ab:8b:e5

Total number of flogi = 5.


============== 3140 Head A with Ontap 8.0.1RC1 7-Mode

3140A*> rdfile /etc/rc

hostname 3140A
ifgrp create single c10g-a e3a e3b
vlan create c10g-a 11 26
ifconfig c10g-a-11 xxx.xxx.xxx.xxx netmask 255.255.255.0 partner c10g-b-11
ifconfig c10g-a-26 xxx.xxx.xxx.xxx netmask 255.255.255.0 partner c10g-b-26
route add default xxx.xxx.xxx.xxx 1
routed on
options dns.enable off
options nis.enable off
savecore

3140A*> sysconfig -v 3

        slot 3: Dual 10G Ethernet Controller CNA SFP+
                (Dual-port, QLogic CNA 8112(8152) rev. 2)
                e3a MAC Address:    00:c0:dd:15:da:b8 (auto-10g_sr-fd-up)
                e3b MAC Address:    00:c0:dd:15:da:ba (auto-10g_sr-fd-up)
                Device Type:        ISP8112
        slot 3: Fibre Channel Target Host Adapter 3a
                (Dual-channel, QLogic CNA 8112 (8152) rev. 2, )
                Firmware rev:     5.3.4
                Host Port Addr:   050001
                Cacheline size:   16 
                SRAM parity:      Yes
                FC Nodename:      50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
                FC Portname:      50:0a:09:83:99:ab:8b:e5 (500a098399ab8be5)
                Connection:       PTP, Fabric
                SFP Vendor Name:  FINISAR CORP.
                SFP Vendor P/N:   FTLX8571D3BCL-QL
                SFP Vendor Rev:   A
                SFP Serial No.:   AH60DND
                SFP Connector:    LC
                SFP Capabilities: 10 Gbit/Sec
        slot 3: Fibre Channel Target Host Adapter 3b
                (Dual-channel, QLogic CNA 8112 (8152) rev. 2, )
                Firmware rev:     5.3.4
                Host Port Addr:   880001
                Cacheline size:   16 
                SRAM parity:      Yes
                FC Nodename:      50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
                FC Portname:      50:0a:09:84:99:ab:8b:e5 (500a098499ab8be5)
                Connection:       PTP, Fabric
                SFP Vendor Name:  FINISAR CORP.
                SFP Vendor P/N:   FTLX8571D3BCL-QL
                SFP Vendor Rev:   A
                SFP Serial No.:   AH60DDE
                SFP Connector:    LC
                SFP Capabilities: 10 Gbit/Sec


3140A*> fcp show adapter

Slot:                    3a
Description:             Fibre Channel Target Adapter 3a (Dual-channel, QLogic CNA 8112 (8152) rev. 2)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname:             50:0a:09:83:99:ab:8b:e5 (500a098399ab8be5)
Standby:                 No

Slot:                    3b
Description:             Fibre Channel Target Adapter 3b (Dual-channel, QLogic CNA 8112 (8152) rev. 2)
Adapter Type:            Local
Status:                  ONLINE
FC Nodename:             50:0a:09:80:89:ab:8b:e5 (500a098089ab8be5)
FC Portname:             50:0a:09:84:99:ab:8b:e5 (500a098499ab8be5)
Standby:                 No


Hope this helps... Perhaps updaint you firmware might help....  I did not specifiy and vlans for the CNA for the FCOE on the netapp.

JUSTIN LOUCKS Tue, 10/12/2010 - 05:27
User Badges:

I appreciate you putting this out here. I do have one question from the configs post above. In the Nexus 5k, you appear to use VLAN 1010 as your FCoE VLAN which is then mapped to VSAN 1000.  I see where you have all of the configuration for VSAN 1000 done, but in the NetApp, you do not appear to have VLAN 1010 configured, only 11 and 26.  If this is due to cleansing your configs before posting, I understand, but if not, please explain where that VLAN definition is on the NetApp side as I think that is what I'm missing.


I'm probably going to go back to NOW support and try again later this week. In ONTAP 8.0.1rc, the adapters support all functionality like a true CNA. You can even run iSCSI and CIFS/NFS traffic across them as they show up as actual Ethernet interfaces with the ifconfig options available.  Unfortunately, in ONTAP 7.3.4 they are seen by the system but only for FCoE access which is fine except that the ability to configure the Ethernet side seems to be absent.


If I come up with a solution for mine, aside from another ONTAP upgrade, I will be sure to post it here...

davidhirt Tue, 09/27/2011 - 06:04
User Badges:

Did you ever figure this out, how the fcoe vlan gets configured on the filer side?

JUSTIN LOUCKS Tue, 09/27/2011 - 06:26
User Badges:

Negative. We essentially came to terms with the fact that the adapter is recognized in 7.3.4 but doesn't have enough functionality to use it. We will be upgrading the system soon and running a recent version of ONTAP 8 which has full support for the adapter and we hope to get it going then for iSCSI and FCoE.

Actually, it looks like the difference here is that the working config by kbhagat has only 2 interfaces per FAS controller. In Justin's config he has 4 ports per FAS controller. We have the same issue. apparently you can not create a VPC accross two Nexus 5K switches with more than 1 port each *IF* and only *IF* you are using FCOE. See page 15 of the document below.


http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572835-00_NX-OS_vPC_DG.pdf




Now, here is my question. Can you create a regular port channel accross the e1a and e2a ports (PC101) and a regular port channel accross the e1b and e2b ports (PC102). Then a VPC accross PC101 and PC102??


Would this work???

Actions

This Discussion

 

 

Trending Topics: Storage Networking