Connecting UCS to IBM DS4700, FC zoning

[Joe Costello]

I am trying to accomplish connecting my UCS to an older SAN that I have to migrate data off.  I currently have a UCS system that is connected to an EMC VNX using 8G FC.  The zoning for this is configured in my UCS.  All works well with the connection to the VNX and I have dual active/active paths working. What I am trying to do is connect a second SAN to my UCS system, a IBM DS4700.  This DS4700 currently has a Brocade FC Switch connected to it, which then connects to a IBM bladecenter chassis.

I configured a second VSAN for the IBM array and configured the port on my fabric interconnect with that VSAN.  I went into the IBM array and I could see the UCS WWN for my HBA, so I connected it to my UCS.  When I did this, I had an unexpected result.  My VMWare ESXi host on the UCS could see all the datastores on the IBM DS4700 BUT lost connection to all the ones on the VNX.  Also, on the IBM DS4700, all of my paths failed over to the controller that had the host connection from the UCS.  I tried to fail them back over to the other controller but they wouldn't move until I disconnected the UCS from IBM DS4700.  Once disconnected from the UCS, everything went back to normal on both sides.

Does any one have an idea on how I am setup wrong here?  The ESX host on the UCS could see the datastores on the IBM so it is somewhat correct but it caused all sorts of problems.

Thanks in advance.

[Joe Costello]

I was thinking about this more and I guess I just don't know FC enough to understand this issue. Can a storage array connect to multiple hosts on different FC zones?  Or can an array only talk to one zone at a time?  Try to figure that out first..

Thanks in advance for any help.

[Raghavendra Barengala Sripada Rao]

Is the FI configured in switched mode? and Is the IBM array firmware supported with UCS system you're running on?

Also what is the failover mode you have set on VNX? Failover mode 4 (AULA) or 1? Try to match failover mode across all hosts to be the same.

And also do you have different HLU for VNX datastores and IBM datastores?

On the FIs do you have different FCoE VLAN for all the VSANs you have created?

It's an interesting problem, so trying to understand in detail.

Thanks,

Rags

[Joe Costello]

I'll see if I can answer your questions correctly:

1. Yes the FI is in switched mode.  It says "FC: Switch"

2. I have no idea if the IBM firmware I have is supported on the UCS, where do I look this up? Likely not as it is a few years old.

3. The VNX to UCS connection is active/active.  I followed instructions from this youtube to configure it:

http://youtu.be/i8Hz2tgpv_Y

4. The IBM and VNX do have LUN numbers that match.  Is that an issue?  They both have something like 1-20 LUN IDs. Figured that they were on seperate VSANs?

5. I have 2 VSANs that are associated with different ports on my FI.  One vSAN is assoicated with the VNX on 2 ports, One VSAN is configured for the IBM on a different vsan number.

Thanks for the help.

Joe

[dynamoxxx · Accepted solution]

Joe,

i don't understand why did you have to create another VSAN just for your IBM array ?  Simply connect FC ports from DS4700 to FI, make sure those ports are in the same VSAN as your VNX and then zone DS4700 to your ESX cluster and sVmotion away. ( I assume you are trying to migrate off DS4700). I think you are making it more complicated than it should be.

@dynamoxxx

[Joe Costello]

I don't know why I did it either. Just thought it made sense to create a second vSAN for a second physical SAN.  I will try setting it up as you suggest and see what happens.  Might have to wait a few days to test this because the last time I attempted it, the IBM SAN did a failover to the controller that I connected my UCS to.  Couldn't use the other controller until I disabled the port on the UCS that connected it to the IBM.  Odd behavior which makes me less confident to try it again during business hours.

Thanks for the advice though, I will try that and let you know.

Joe

[Raghavendra Barengala Sripada Rao]

As dynamox suggested just have only 1 VSAN/Fabric. Also when it comes to HLU for LUN ID it need not be different, but it's good to have a different HLU for LUNs from two different Array's for better understanding.

Also just curious to know why UCS was implemented in switched mode rather than End Host mode? Switched mode is not the recommended way!! Just curious to know.

[Joe Costello]

I thought switched mode was used to enable FC zoning from the UCS and not a seperate device.  I'm not sure, we had some help setting it all up and that was how we did it.  I'll have to look at the settings and do some research to see what works best for us.

Thanks

[dynamoxxx]

if you have no plans of connecting UCS FI to Brocade then you you will need to do all your zoning on FI, hense switched mode.

@dynamoxxx

[Joe Costello]

SO, I used the same VSAN to connect the UCS to the IBM and it worked.  The caveat is that the LUNs on the IBM that have a path to controller B (host port 1 on controller b of the IBM is the port that I am using to connect to the UCS) are the only ones that can see both the IBM and EMC storage from the UCS.  To fix the issue, I am planning on running a second FC connection to the IBM from the UCS, this one will plug into the A side of the IBM and should allow all LUNs to be available to the UCS.  Didn't reliaze that the UCS wouldn't be able to see all the LUNs running on controller A unless I was plugged in on that controller as well.  Does this make sense to you as well?

Thanks for the help.

[dynamoxxx]

it makes perfect because that array is active passive from LUN ownership perspective (just like VNX). That's why you have to zone your host to SPA and SPB to have access to both controllers. In case of trespass (where LUN transitions from SPA to SPB or the other way around), the host still has access to the LUN. Always make sure to zone each HBA to SPA and SPB, for example:

Fabric A

Zone 1 -  Host  HBA 1 / SPA0

Zone 2 - Host HBA 1 / SPB2

Fabric B

Zone 1 - Host HBA 2 / SPB0

Zone 2 - Host HBA 2 / SPA2

Does it help ?

@dynamoxxx

[Joe Costello]

SO, I thought I had this figured out but I am still running into an issue.  I created a host group on the IBM SAN with the HBAs from both my UCS and my other host.  From the UCS, I now have 2 paths, one to the A side and one to the B side.  For the host group that I have setup, it looks to work.  However, when I add the UCS HBA to the older host group (the one that contains the HBAs for our bladecenter) that has access to all the the LUNs mapped to the other hosts, all the LUNs failover to HBA B on the SAN.

You mentioned creating 2 different zones above, do you mean 2 different vSANs? Or is a zone just the connection to a second SAN?

I looked at it a little more.  My guess is the issue is when I add hosts from 2 different zones into the same host group?  When I add one host to SAN with 2 paths, I can see both the A side and B side normally. (for both UCS and bladecenter)  BUT when I put them both in the same host group, that seems to cause problems and everything fails over to the B side.  Can I not do that?  I want the UCS to see all the older datastores so I can vmotion them off.  I have a seperate host group that work with just UCS servers OR just IBM servers.  Guess I am missing something.

Thanks in advance for the help.

[Joe Costello]

Almost seems like I need to create multiple LUN mappings on the IBM SAN?  One mapping to the UCS hosts and one mapping to the IBM hosts?  I don't see that as an option though.

[dynamoxxx]

i am talking about zones, not VSANs. One VSAN can have multiple zones.

I do not have experience with IBM storage arrays but all mid-tier arrays are very similar. Can you establish this type of connectivity ?

UCS FI FABRIC A:

IBM DS4700 - Port A

IBM DS4700 - Port B

UCS FI FABRIC B:

IBM DS4700 - Port A

IBM DS4700 - Port B

If you can establish this type of connectivity, then you can create zones just like i provided above. Once UCS hosts are logged-in to your storage array , add them to the same host group as current bladecenter servers. One some vendor arrays (EMC VNX/VMAX for example) you can create a brand new host group and add the same LUNs that are already presented to the new host group.  Ultimate goal is to ensure that new servers can see your datastores and then use sVmotion to move VMs to new LUNs.  Same ESXi version on bladecenter servers and UCS ?

@dynamoxxx

[Joe Costello]

My DS4700 has 2 host ports on each controller, so a total of 4 host ports.  2 of them are going to a brocade FC switch, the other 2 are going as follows:

UCS FI FABRIC A:

IBM DS4700 Controller A - Host Port 2

UCS FI FABRIC B:

IBM DS4700 Controller B - Host Port 2

Will this not work?  Thats all the host ports I have available on the controllers.

Verisions of ESXi differ slightly, older is on ESXi 4.1, UCS is on ESXi 5.1.I know that my UCS hosts are connected to the IBM because I have setup test LUNs on a seperate host group on the DS4700.  The UCS can see both LUNs and each LUNs is on a different controller, one on the B side and one on the A side.  So, it looks good from a connection standpoint.  WHen I go to add the UCS host to the host group with all the other LUNs, I am still getting failover to the B side.