Fabric 6248 with MDS

[anis_cisco]

Hello all,

We are in stage of inter-connecting single IBM V3700 SAN box with dual FI 6248 (Redundant) via single Cisco MDS 9000 SAN Switch with single UCS 5108 chasis.

We have four blades B200 M3 in UCS 5108. Blades DO NOt have local hard drives. Our requirement is to install hyperV/Windows on IBM storage.

I have completed the physical connections from IBM to MDS & MDS to FI 6248 & FI 6248 to dual Core 6509 VSS enabled.

I dont have issues with FI 6248 & LAN part, i have doubts on FI 6248 with SAN kindly advise.

FI 6248 are in default mode (End Host). Do i need to change FI on Switching mode ?
FI Unified ports connected to MDS are configured as FC Storage, Do i need to change to FC Uplink Ports ?
How does vNIC, vSAN (how many vSAN do i require), vHBA & Service Profile works to each other ? can any one advise please ?
What steps do we need to configure in FI & MDS so that we can utilize SAN Storage for hyperV/Windows installation ?

This is my first deployment of UCS 5108, apologize to asking basic questions.
 

Regards,

[Walter Dey]

UCS Fabric Interconnect in FC EHM is perfect !

You have to enable NPIV on the MDS however !

flogi will happen on the MDS ! zoning is done on MDS !

You would create different VSAN's for fabric A resp B.

You seem to configure boot from SAN; therefore you have the choice of 2 vhba's, one in each fabric, which are used for accessing boot lun's as well as data lun's.

In the case of multiple FC links per fabric between FI and MDS: F-port trunking channeling is recommended.

------------------------------------------

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/os-install-guides/windows/b_B-Series_Windows_Install/BSERIES-WINDOWS_chapter_011.html

https://supportforums.cisco.com/discussion/12238936/multiple-san-uplinks

https://supportforums.cisco.com/discussion/12233821/building-windows-2012-service-profile

[anis_cisco]

Hello wdey,

Thank you for your reply it helped me towards step ahead.

I went through some video tutorials & documentation as well which cleared the UCS work flow for me.

I have few questions kindly guide me please,

1- what would be the impact of multiple vSans vs Default vSAN, i am planning to use default vSan in both FI-6248 & i can't find any issue with that, kindly advise.

2- WWPN pool, should i create two different WWPN pool 1 for FI-6248-A & other for FI-6248-B ? I have only one UCS 5108 chasis, can't we configure a single WWPN Pool for both FI-6248 ? what is the best practice ? i can't find any issue with single WWPN pool. kindly advise.

3- Incase, if i will configure 1 WWPN Pool would i face any issue in zone mapping into MDS ? As per my understanding there won't be any issue as Zone is just the mapping of initiator (Blade Server) & target (IBM Storage). Keep in mind i have single Cisco MDS SAN Switch NOT redundant. kindly advise.

4- While creating Service Profile what Adapter Policy should i select in Dynamic vNIC menu ? I want to install Windows on Hyper V environment but i cant find hyper V option there. It has linux, VMWare, VMWare passthrough, window, default & few more but no HyperV ?

Regards

[Walter Dey · Accepted solution]

1- what would be the impact of multiple vSans vs Default vSAN, i am planning to use default vSan in both FI-6248 & i can't find any issue with that, kindly advise.

Please don't use default vsan (or vsan nr 1); best practise: create different VSAN's for fabric A and B; e.g. vsan 10 for A, vsan 20 for B

2- WWPN pool, should i create two different WWPN pool 1 for FI-6248-A & other for FI-6248-B ? I have only one UCS 5108 chasis, can't we configure a single WWPN Pool for both FI-6248 ? what is the best practice ? i can't find any issue with single WWPN pool. kindly advise.

Best practise: use different pwwn pools for fabric A and B; useful for trouble shooting; generally speaking: all pools should also encode a UCS domain id (your start with 1); just in case you have in the future another one ! see attachment.

3- Incase, if i will configure 1 WWPN Pool would i face any issue in zone mapping into MDS ? As per my understanding there won't be any issue as Zone is just the mapping of initiator (Blade Server) & target (IBM Storage). Keep in mind i have single Cisco MDS SAN Switch NOT redundant. kindly advise.

Even with one MDS, you create virtual dual fabric with vsan 10 and 20; zoning is done per vsan, zoneset and activation as well !

4- While creating Service Profile what Adapter Policy should i select in Dynamic vNIC menu ? I want to install Windows on Hyper V environment but i cant find hyper V option there. It has linux, VMWare, VMWare passthrough, window, default & few more but no HyperV ?

You don't create dynamic vnics ! this only applies to VM-FEX, resp. SR-IOV. Hyper-V means Adaptor Policy = Window

Good luck

Walter.

[anis_cisco]

Hi walter,

I went through above all as mentioned & i am able to apply service profile template successfully on blades without any error on chasis and FI.

Next step was to configure MDS with Zone between Initiator (Blade) & target (SAN) WWPN which deployed successful as well, but boot from SAN didn't happened.

I can see the Blades WWPN (vsan 200, 202) & SAN WWPN (vsan 1 - As per storage team, they can't configure vsan) in MDS but from storage if i add Blade WWPN its not recognizable.

Attached are the few outputs from MDS, FI-A with screen shots any idea where is the issue. We are trying to boot from IBM V3700 storage.

[Walter Dey]

I will have to study this in detail, but one thing is quite clear !

The 4 IBM storage ports are now in VSAN 1; this is not ok; 2 of them most likely have to be in vsan 200, 2 in vsan 202; this is not a problem with any attached device; the vsan tagging will be removed outound, resp. added inbound.

Here my 2c

- Naming: I would encode fabric in the zone name: HyperV-Zone-1-A resp B

- IBM-Connected-Bladex for IBM storage port alias is misleading (wrong); the outgoing loadblancing from UCS to IBM is round robin

- I assume you will do the identical configuration for Fabric B (zoning... are per VSAN)

- I would use device-alias, instead of fcalias. The latter are per VSAN, the former per fabric; this is of course cosmetic

- I would also use enhanced zoning (I attach a few useful CLI); cosmetic

Please post output of, once the IBM ports are in VSAN 200 / 202, and the zoning is changed accordingly

MDS CLI: show zoneset active vsan 200 resp 202

====================================================================================================

-------------------------------------------------------------------------------------------------------------------------------------------------

device-alias mode enhanced

  device-alias name VPLEX-Bal16-11-B-WAN-0-a pwwn 50:00:14:42:b0:03:41:20

device-alias commit

-------------------------------------------------------------------------------------------------------------------------------------------------

vsan database

vsan 100 name "VPLEX Front End"

vsan 101 name "VPLEX Backend"

………

  vsan 100 interface fc1/1

………

-------------------------------------------------------------------------------------------------------------------------------------------------

fcdomain domain 10 static vsan 100

fcdomain domain 12 static vsan 101

system default zone distribute full

system default zone mode enhanced

zone mode enhanced vsan 100

zone mode enhanced vsan 101

-------------------------------------------------------------------------------------------------------------------------------------------------

zone name host_vios2-16_h1_a--VPLEX-Bal16-11-B-FE-0-a vsan 100

member device-alias VPLEX-Bal16-11-B-FE-0-a

member device-alias host_vios2-16_h1_a

zoneset name VPLEX-Frontend-a vsan 100

member host_vios2-16_h1_a--VPLEX-Bal16-11-B-FE-0-a

zone commit vsan 100

-------------------------------------------------------------------------------------------------------------------------------------------------

[anis_cisco]

hi Walter,

As suggested, i added 4 ports of SAN in vsan 200 & 4 ports in vsan 202 & looks like things are fine now but still i am facing issue on IBM side. IBM SAN is not recognizing blade's WWPN.

About zone naming I didn't change the zone names i kept them same & only correct the WWPN numbers, attached is the complete output with show zoneset active. Kindly advise if some thing is missing.

Moreover, today i opened case with Cisco & TAC engineer confirmed that there is no issue in configuration from our side, there some thing must needs to be configured in IBM SAN.

I am thinking of compatibility doubt between UCS & our SAN (IBM Storwize V3700)

As per following link from IBM, it seems IBM Storwize V3700 is compatible for Cisco UCS B-Series design with FI-6248 & MDS 9148

http://www-03.ibm.com/systems/support/storage/ssic/interoperability.wss

But as per following link from Cisco it seems V 3700 is not supportable by MDS 9148.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx/Matrix8.html

Can you confirm the compatibility ?

Regards,

[Walter Dey]

Hi

Good progress !

I see that one IBM port is not working; fabric A (Vsan 200) has 4 links, fabric B (Vsan 202) only 3

zone name HyperV-Zone4-FIB vsan 202

* fcid 0xcc0308 [pwwn 20:12:00:25:b5:11:01:4f]

pwwn 50:05:07:68:03:10:7d:09 <=============== is missing !

What do you mean by

IBM SAN is not recognizing blade's WWPN. ?

you don't see the Lun on the IBM storage ?

I can tell you that zoning is done correctly !

Q. is lun masking/mapping done properly on IBM storage ?

====================================================================

according to UCS V.2.2 interop matrix

http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/interoperability/matrix/r_hcl_B_rel2-21.pdf

DS 3500, DS 4800, DS 5300, DS 8100,
DS 8800, SVC, Storwize V7000, IBM
N-Series, XIV(Gen 2), XIV(Gen3)
is supported !

[anis_cisco]

Yes,

The fourth interface is not physically not connected yet that's why its not recognizing, we can ignore it don't worry.

Basically the requirement is to boot blades from SAN. Hence for this we added Blade's WWPN into SAN. But IBM SAN is not recognizing the blade's WWPN automatically & even manually as well. IBM showing the WWPN are not recognizable & shows offline.

For this issue i opened case with Cisco & engineer confirmed there is no issue on Cisco side, something needs to be done in IBM side.

IBM engineer said there is nothing much to be configure on V3700.

For this i want to verify the compatibility of IBM SAN & Cisco UCS B Series first.

[Walter Dey]

OK

Basically the requirement is to boot blades from SAN. Hence for this we added Blade's WWPN into SAN. But IBM SAN is not recognizing the blade's WWPN automatically & even manually as well. IBM showing the WWPN are not recognizable & shows offline.

Maybe I understand; IBM doesn't see the pwwn of the initator coming from UCS blade ? correct ? and therefore they claim being unable to do their lun masking / mapping.

The output of show zoneset active clearly says, that the hba are flogi into the MDS fabric. So this is ok.

It should be possible to register the initiator pwwn manually. The same problem I have seen many times with EMC.

After all; what is the version of the MDS ? if it's old, an upgrade to a actual version might be highly recommended.

[anis_cisco]

Yes that's our problem. That IBM is not recognizing initiators WWPN.

My MDS is on latest version m9100-s3ek9-kickstart-mz.5.2.8d.bin & m9100-s3ek9-mz.5.2.8d.bin

So what you did for EMC when you faced same issue ?

Regards,

Anis

[Walter Dey]

Hi Anis

OK, fine; I'm fairly convinced that this is a IBM storage array setup issue

There is a manual procedure, which most guys didn't know. One has to remind them, that it should be possible to configure a storage array, without having the actual hardware connected, giving them the list of initator pwwn.

Tell them, that there is no OS installed on the blade, that you have to first install this over the network; nevertheless the vhba is working according to the setup in the service profile.

Walter.

PS. Not quite the same issue, but interesting

https://supportforums.cisco.com/discussion/12200351/b200-m3-san-boot-windows-2012-issues

[anis_cisco]

Hi Walter,

So I think for now we should wait for IBM guys & let see what is there finding on this issue.

I will update the post here as soon as i will hear back from IBM.

Thank you so much to being kind for my queries & your guidance :)

Appreciated.

Regards,

[anis_cisco]

Hi walter, Regarding above topic IBM team is still working with IBM support team & didn't come up with solution yet. I need to know that the PWWN nunbers are assigned to blades dynamically from pool (assigned in service profile). Incase if blades restarted the pwwn number will be change ? if pwwn will change then it will effect in zone configuration on MDS Switch. How to fix the pwwn to each blade? Regards

[Walter Dey]

Hi

Once the service profile has been created, all the values it took out of the pools (UUID, MAC, pwwn, nwwn) don't change anymore; any operations like disassociate SP, associate SP,.... will leave this values unchanged.

You can see the values by inspecting the vhba's; resp. the corresponding pools.

Good luck

Walter.