UCS and NetApp (fiber connect)

[randerson]

UCS Mgr - 2.0 (2q)

FIs - 6248

IOM - 2208

NetApp - 2040

I've read a bunch of different discussions on here about the best way to connect a mass storage device to the UCS FIs. In our case, we have a NetApp filer which mainly services a virtual environment via iSCSI. As its a smaller filer, it only has 1gb ethernet NICs and 4gb fiber interfaces. The 1gb ethernet interfaces are all in use (with VIFs). We have a virtual environment running with iSCSI back to the NetApp but it has to traverse our core 5548s and downstream 2248s, which is what the NetApp ports are connected to.

Right now we have available fiber ports on our NetApp and I want to use those for UCS backend storage. Most articles I have read have recommended that the FIs (6248s) be VPCed up to the core (which we're doing to our 5548UPs, works great) and then you have MDS fiber switches connected to the 5548s and/or the 6248s. I tried to directly fiber connect the NetApp to the 6248s but it doesn't recognize it (I changed the port to a "Storage FC Port" and restarted the 6248)

Since we have such a limited fiber deployment, I likely won't be buying MDS switches in the near future so it would seem that the next best approach would be to fiber connect our NetApps to the 5548UPs and maintain all of the fiber infrastructure from there. Its my impression that you cannot fiber connect your storage directly to the UCS environment due to the 6248s only have a subset of the MDS code for supporting fiber environments (ie. vsans, zones, etc.)

Is this a correct assessment?  Can I directly fiber connect our NetApp to the 6248s? It would be an ideal situation since the UCS would be the only devices using fiber to connect to the NetApp. Can anyone direct me to a whitepaper or configuration guide for setting up the 5548s and consequently 6248s) for fiber connections?

Thanks in advance!

[Brian Morrissey]

Hi Ross,

Is this similar to what you are looking to accomplish (http://www.cisco.com/en/US/partner/docs/solutions/Enterprise/Data_Center/Virtualization/flexpod_vmware.html)?

[randerson]

Thanks for the response Brian.

Yes, that is very similar to our setup except our NetApp ethernet NICs currently connect to downstream 2248s (uplinked to the 5548s). That is not an issue as all of the iscsi traffic is isolated to the 2248s at the moment.

It appears that with regards to the fiber connections, the confguration needs to be created on both the 5548s and the 6248s, independent of one another - in other words, the fiber configuration on the 5548s does not automatically get transferred to the 6248s (such as with VTP, for example).

That being the case, why do the ports on the 6248s even have a "Storage FC" option? I can understand the Uplink FC ports, which would connect to an upstream MDS switch or the like.

[Brian Morrissey]

Hi,

Yes you will need to configure the uplinks on both the 5548 and 6248.  However, the zoning will be configured on just the 5548 due to the way the 6248 currently works.

The Storage FC port is used for direct-attach storage like in the example above, the Uplink FC would be used if you connected the Netapp to the 5548 instead.

[randerson]

Forgive my confusion, but in your second paragraph you say the Storage FC port is used for direct-attach storage like in the example above (in the flexpod VM archictecture example the storage only connects to the 5548s). So why can I not direct fiber attach the NetApp to the 6248s then once I configure all of the fiber environment on the 5548s and the 6248s?

This is essentially what I am trying to do but I thought there was a limitation where I HAVE to fiber connect the storage device to the 5548s, and NOT to the 6248s. In what sort of scenario could you direct fiber attach storage to the 6248s?

[Brian Morrissey]

Hi Ross,

My mistake, you are right it is going to the 5548 in the picture above.  This one is better (http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/whitepaper_c11-702584.html):

You are not required to fiber connect the storage device directly to the 5548.  If you wanted to you could directly attach the storage to the 6248s as long as you have an upstream MDS/Nexus for zoning.  The ucs fabric interconnect must also be set to switching mode for this to work which requires a reboot.

[randerson]

Ok, that makes sense - I don't want to be in switching mode however because you lose the other benefits of the end-host mode (can't remember all of those benefits, but I know switching mode is only useful for a few select situations).

I'll see if I can get the 5548s setup and directly fiber connect the NetApp there -- thanks for the help Brian!

BTW - the picture above is exactly what I had hoped to do (fiber connect NetApp directly to the 6248s) but I don't think I want to be in the FC switch mode.

[e.nieuwstad]

If you want the Netapp directly connected to the FI you will need to be in FC switch mode see the link below

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/141/UCSM_GUI_Configuration_Guide_1_4_1_chapter4.html#concept_495162A236314097A8F78B339C697C60

you could hook up the FI to the 5548, enable npiv on it and connect the Netapp to the 5548 and do everything from there. However it requires the storage license which is quite expensive even more than a pair of MDS 9124

If you don't have a SAN network presently there is no benefit of NPV mode anyway

[randerson]

Just a sec - you have to have the storage license on the 5548s in order to fiber connect an end device or is the license only required when using NPV mode?  I assume it's the latter. Is it possible to do what I need to do without NPV mode?

Basically, all I want to do is fiber connect my storage device so that there is more potential bandwidth going to the UCS environment. That would mean connecting our NetApp to the 5548s - so the only way to connect the NetApp to the 5548s is by enabling NPV mode and getting a storage license?

Thanks!!

PS. I think I see now - license N55-8P-SSK9 is required to enable FC and/or FCOE ports (8), regardless of NPV mode.

[e.nieuwstad]

yes you are right to use it for anything regarding FC storage you would need a license. The license forFC in the FI is included off course