Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

UCS Design Question

Hi all,

i am having trouble designing a 2 Site UCS-System. We plan to use two Blade-chassis, one for each site. We want to use two Fabric-Extenders per Chassis. Now we want to have two Fabric-Extenders in each site. Lets say the two sites are <100 m as an assumption. In each site we have one nexus 5548up, an from that we connect fcoe (Netapp) and ip (Catalyst 6500 Layer-3, 2 Sups on each Cat).

dc3-ucs.png

Now i would appreciate any help on how to connect the for interconnects together.

Lets say from left to right it's interconnect1-4. I woul connect interconnet 2 with 2 Links to interconnect 3 and inetrconnect 1 with 2 Links to interconnect 2 and interconnect 4 with 2 links to interconnect 3. But somehow i know, that this can't be right. Or does the Clusterlogic allow that?

To make it short: I don't know :-)

I know that the interconnects 'see' each other even (or only) over the Nexus 5548.

It would be great if someone could solve the Problem before i go to cisco networkers in London at the end of January.

Regards

Michael

Everyone's tags (3)
8 REPLIES
Cisco Employee

UCS Design Question

Michael,

I'm a little concerned about this design.  You're leaving a single N5K on each site, which is a single point of failure.  Do you really want your UCS environment inaccessible because of a potential problem on your lone N5K?

Additionally, each UCS system is made up of a pair of Interconnects and then multiple chassis.  You can't "add" more interconnects into the mix like this.  Each site would maintain it's own separate UCS cluster.  It's not possible today (due to multiple reasons such as latency of FCoE & distance limitaitons) to run a cluster across sites.

Regards,

Robert

New Member

UCS Design Question

Okay i got that, but the 'Buying resistance' of a second n5k is quite large, so we'd go with that spof.

Anyway, i definately statet that there's <100 m between the sides, so the latency shouldn't be the showstopper. And in some advance of time we might want to add more chassis to each side. But concerning the HA-Design, we'd like to have only the n5k as a spof.

Actually, this is of course supposed to be a Flex-Pod. Storage is Netapp and we'd run 1000v and Vmware on the chassis/blades, but thats not the point of my Post.

It still remains: How to connect the FI's without seeing the n5k as a spof.

Regards

Michael

New Member

UCS Design Question

A UCS Cluster can only contain 2 FIs.  Given your setup, FIs 1 and 2 (Site 1) will be interconnected and will be managed together via UCSM.  The same will be true of FIs 3 and 4 in site 2.  These two UCS clusters won't "see" each other at all.

And as Robert mentioned, stretching a single cluster across physical sites is not supported.  So you couldn't interconnect FIs 1 & 3 and then 2 & 4, for instance.

Since a cluster will be contained to each site, an N5K failure won't interrupt the functioning of the UCS cluster, only the network connectivity to that cluster.  At this point, it's a more classic redundancy problem.  You can add extra switches to the mix, or utilize redundant links and connect the UCS to both N5Ks (STP or vPC).

Cisco Employee

UCS Design Question

Keep in mind he's using his N5K as his storage switch, so any SAN/LAN boot blades would fail (crash) upon an N5K failure, not just network connectivity to the cluster.

Robert

New Member

UCS Design Question

Absolutely, storage would need redundancy as well.  Without multi-hop FCoE, though, it's either direct attach to the FIs (bypassing the N5Ks) or using FC from UCS to 5K.

The original question about interconnecting FIs has a straightforward solution, since there's really only one possibility.  It's up to him to decide what he wants to do about redundancy to mitigate an N5K failure.  Just wanted to give him some options.

New Member

UCS Design Question

Hi All,

thanks for the replies!

The n5k ist absolutely clear, and what is not shown in the diagram ist that storage is a cluster too (Streched Metrocluster).

What if we'd go for only 2 FI's ? One in each site. Would that be a solution for havin a ucs-cluster in the two sites? This way we'd hav 2 n5k, 2 Fis, 4 Fexes and all ist good? This might be it, or am i missing something?

Regards

Michael

New Member

UCS Design Question

Michael,

Did a little digging and I'm going to recant the statement about lack of support over the physical distance.  Looks like that as long as you abide by the distance limitations (100m between FIs, 300m between FI and IOM) you would be ok.  Here's another topic that discusses a similar issue:

https://supportforums.cisco.com/message/3193163

So, in theory, you could use an FI in each site and stretch the cluster (under 100m).  Keep in mind that you would need to dual-connect all chassis to both FIs.  So chassis in site 1 would have to run links to site 2 FI, and same in the other direction.  It's do-able, but still may not be the best solution.

Hope this helps.

Cisco Employee

UCS Design Question

Again, stretching a UCS cluster across sites via , dark fibrek, MPLS, MetroWAN or whatever is not supported.  Only direct attach is, which has distance limitations. 

You might be able to "MacGuyver" it and get it to work, but if you hit any issues TAC would not be able to assist.

Robert

1556
Views
0
Helpful
8
Replies
CreatePlease login to create content