IBM Blade Switches - Can I externally cross connect them?

Unanswered Question
Sep 23rd, 2010

Hello,

I have a customer with an IBM blade center, it has 2 Cisco blade switches installed, (3112)

I have connected the inside interfaces of a pair of ASAs and CSSs in failover configuration, to the external ports on each switch, but as each switch is effectively standalone, the ASA failover doesn't work due its inside interfaces not 'seeing' each other, as they are in different switches.

Would I be ok in patching a cross-over cable between both blade switches, and this would then allow ASA failover to work properly.

Or would this link create a loop with the blade servers?

Any help would be grately appreciated

Regards Tony 

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4 (9 ratings)
Loading.
Jon Marshall Thu, 09/23/2010 - 10:07

[email protected]

Hello,

I have a customer with an IBM blade center, it has 2 Cisco blade switches installed, (3112)

I have connected the inside interfaces of a pair of ASAs and CSSs in failover configuration, to the external ports on each switch, but as each switch is effectively standalone, the ASA failover doesn't work due its inside interfaces not 'seeing' each other, as they are in different switches.

Would I be ok in patching a cross-over cable between both blade switches, and this would then allow ASA failover to work properly.

Or would this link create a loop with the blade servers?

Any help would be grately appreciated

Regards Tony 

Tony

Are the IBM blade switches connected to any other switches or are they effectively standalone. Usually they blade switches are connected to a distribution pair of switches connected with a L2 trunk so the 2 blade switches do see each other via the distribution pair.

But the blade switches should have an internal interconnect between them in the chassis anyway.

Jon

tholmes@cistek-... Fri, 09/24/2010 - 01:52

Hi Jon,

Thanks for the reply.

In this case no, the CSS and ASA pair are directly connected to the blade switches.

Traffic comes from the Internet to the CSS VIP and on to the blade servers, the blade servers then send data via an ASA FW to a SAN server

(having blade servers directly connected to the Internet was at the customer's insistance)

The blade switches are not internally connected though,

eg. a device in blade switch 1, interface g0/17 in VLAN 4 is not able to see another device in blade switch2 interface g0/17 VLAN 4.

I've seen examples of each blade switch being connected to a pair of distribution switches, and these distribution switches are then cross linked by trunks, I can't then see the difference of cross patching the blade switches with a cross-over cable and remove the need for distribution switches.

Any reason why this wouldn't work, from a spanning tree perspective?

Cheers Tony

Reza Sharifi Fri, 09/24/2010 - 09:08

Hi Tony,

I don't think your solution is going to work since 3112 switches are not stackable. The switches that support cross stack uplinks are the 3750s and 3110s.  Basically you are trying to mimic stacking by connecting the 2 switches together, so ASAs can see both switches as one.

HTH

Reza

tholmes@cistek-... Fri, 09/24/2010 - 09:20

Hi Reza,

I appreciate your response but everyone keeps telling me they are just like 2 Cisco 2950 series switches, except they have internal and external interfaces. (and that VLAN 1 is reserved for iinternal management)

I might just get some downtime and experiment!

Cheers Tony

jasonfmic Fri, 09/24/2010 - 13:02

He doesn't need to stack the switches. He only needs a trunk interface between them.

Reza Sharifi Sat, 09/25/2010 - 06:27

Jason,

Thanks for the info and also clarifying this.  I guess, I misinterpreted Tony's question by thinking he needs to stack them together.

Tony,

I apologize if I misinformed you

Reza

tholmes@cistek-... Sat, 09/25/2010 - 06:34

Hi Reza,

No worries mate, I really apprecaite the feedback, it goes to show these switches are a little confusing and the Cisco documentation isn't that great. The IBM documentation is awful!!!

Cheers Tony

jasonfmic Fri, 09/24/2010 - 13:04

Jon,

These switches do have an internal connection, but it is a management access interface only and not for data traffic.  This internal connection connects to the management modules in the IBM chassis and typically management traffic is actually routed out-of-band through the management modules to the switches while all other data traffic is directed out the external ports on the back.  The switches cannot pass data traffic with their internal connection. In order to accomplish this, he will need to connect the external ports in a trunk configuration or connect both switches to a distribution layer.

Jon Marshall Fri, 09/24/2010 - 15:20

Jason

Thanks for the info regarding the interconnect. I'm obviously a bit out of date on these as the ones i have dealt with do indeed have an interconnect that you often had to disable or you got an STP loop.

Tony, apologies for the incorrect information, hope it didn't inconvenience you.

Jon

jasonfmic Fri, 09/24/2010 - 15:36

Not to worry, there is some ambiguity in the documentation on these. In fact, I have seen what you are talking about on the older IGESM modules. In fact, the newer modules do not allow you to set the access vlan of the internal facing port which is now a layer 3 fast ethernet port that gets configured by the management modules.  This was a source of much grief for me when I migrated our entire data center from the IGESM modules to 3012s. I remember having TAC triple check then check again because of issues we had had previously with the IGESMs.  At the end of the day, I have found the 3012 & 3110G switches to be much more robust. We have had zero issues with them.

tholmes@cistek-... Sat, 09/25/2010 - 06:15

Hi Jon,

Not a problem mate, I appreciate the feedback and as its a forum it generates debate and cooperation, I vaguely recall HP blade switches are able to internally connect but not the newer IBM ones, I hate these things because there is always a knowledge gap between the blade center VM ware server bods and the network engineer

:-)

Cheers Tony

jasonfmic Fri, 09/24/2010 - 13:00

Are these the only switches in your infrastructure? The ports on the back of these switches are really meant to uplink to your switching infrastructure..

If you require that you use only these switches, then, yes, what you need is a trunk port between the two switches. The external interfaces can be configured as trunk interfaces on each switch and connected to each other.  You could also bond two interfaces on each switch into an etherchannel if you wish to connect more than one interface without shutting it down with spanning tree.  The only odd magic going on with these switches is the way that the management is handled through the AMM modules. Other than that, they are just normal switches with 14 internal interfaces for the blade servers and 4 external interfaces for uplinks.

I'm curious about your use case though. Are you attempting to configure each IBM bladecenter basically as an independent security zone with a pair of ASAs?  Is this a service provider scenario?

Jason

tholmes@cistek-... Sat, 09/25/2010 - 06:26

Hi Jason,

This does make sense to me, as it seems logical to connect these 2 switches externally, directly together. Even if I were to run a pair of links to 2 external switches, say 3750s, then I'd only be moving the inter-link upstream so, VLAN-wise, they'd still 'see' each other.

As for topology, traffic from the Internet passes through a pair of CSSs which connect to the blade switches and on to the blade servers, effectively the blade servers are open to the Internet, (its not my design, as I'd originally suggested a FW in front of the CSS)

The web servers on the blades then need to talk back to a series of SAN boxes, these ARE protected by a pair of ASAs, hence the need to plug HA CSS inside interfaces and HA ASA outside interfaces into the blade switches, with failover running these devices need to send and receive their keepalive traffic. Which is why I want to trunk them together.

Thanks for all your help Gents, as a freelance engineer, its great to be able to bounce ideas off other like minded engineers

Regards Tony

tholmes@cistek-... Tue, 10/12/2010 - 03:36

Hi,

I visited the DC this weekend and patched both the blade switches to a pair of 3560s (which are themselves connected by a trunk link). I then configured an isolated VLAN 10 on the 3560s to conect the CSSs and ASAs into the same VLAN as the blade servers.

I have not used trunks to connect the blade switches to the 3560s

So each blade servers is dual linked internally to the blade switches, which are both patched into the distribution pair of 3560s.

Alas, running a continuous ping on a blade server and physically pulling out the top blade switch, resulted in a loss of connectivity to the Internet

I will get the blade server NIC teaming checked, as they should have resumed sending traffic over the second blade server!!

Cheers Tony

Actions

This Discussion