Switched Module Designs

lamav · ‎05-06-2010

The scenario is a data center sever farm...

A good rule of thumb to prevent excessive cross link traffic in a switched access /routed agg design is to make each agg switch the root and hsrp primary for each vlan. Then alternate this for each vlan so that you can use both uplinks. Simple.

However, in a design that requires service modules, only one can be active, so the recommendation according to Cisco Data Center Design Guide 2.5 is to make one agg switch (the one that hosts the actve switched module) the root and hsrp primary for all vlans. Fine, but this leaves the other uplink totally idle. A waste.

What workarounds have some of you used for this? Is there a simple workaround?

Perhaps leveraging contexts and making each switched module the active for a certain set of vlans and having those vlans be part of only that context -- and then making the hosting agg switch the root and hsrp primary?

Thanks

Jon Marshall · ‎05-06-2010

lamav wrote:

The scenario is a data center sever farm...

A good rule of thumb to prevent excessive cross link traffic in a switched access /routed agg design is to make each agg switch the root and hsrp primary for each vlan. Then alternate this for each vlan so that you can use both uplinks. Simple.

However, in a design that requires service modules, only one can be active, so the recommendation according to Cisco Data Center Design Guide 2.5 is to make one agg switch (the one that hosts the actve switched module) the root and hsrp primary for all vlans. Fine, but this leaves the other uplink totally idle. A waste.

What workarounds have some of you used for this? Is there a simple workaround?

Perhaps leveraging contexts and making each switched module the active for a certain set of vlans and having those vlans be part of only that context -- and then making the hosting agg switch the root and hsrp primary?

Thanks

Bet you wish once in a while someone else would answer these questions

You've answered your own question really ie. use contexts on the service modules if they support it. Other alternatives -

1) there may be some vlans you do not want to go through the service modules so you can use the other switch for these vlans

2) use a very large etherchannel trunk between the 2 agg switches and accept that there will be a lot of interswitch traffic going between these vlans.

Not sure how if VSS fits in here and whether it could help to be honest. I doubt it because as you say no matter which switch it ended up on only one of the 2 switches actually holds the active service module.

As a final point, the whole load-balancing vlans by using both uplinks and manipulating HSRP active/STP root. I remember a couple of years back i was talkng to a DC cisco guy whose opinion was that by all means do it but in a DC either single link should be able to take the full load for all vlans regardless.

Jon

lamav · ‎05-06-2010

Jon:

Just the opposite, I like that you answer these questions because you seem to have extensive experience in the data center, especially when it comes to deploying firewalls and load balancers in the server farm.

That having been said, I do wish others would give their input, too. I like to hear about different people's experiences.

Im glad I answered my own question -- it makes me feel that I am not missing anything too big. lol

I am in the process of bringing up a new switched access layer for a client who has purchased 4948s. The 4948s run at wire speed with a 96 Gbps backplane. I am trying to convince them that utilizing only 2 of the 4 SFP uplinks is a waste. Using 2 will give them a 24:1 oversubscription ratio. At least with 4 ports, they will have a 12:1 OS ratio. Both OS ratios are assuming that they use both uplinks for alternating vlans, as we discussed earlier. Otherwise, with one active uplink, itll be 48:1 or 24:1 in the latter case.

Jon Marshall · ‎05-06-2010

lamav wrote:

Jon:

Just the opposite, I like that you answer these questions because you seem to have extensive experience in the data center, especially when it comes to deploying firewalls and load balancers in the server farm.

That having been said, I do wish others would give their input, too. I like to hear about different people's experiences.

Im glad I answered my own question -- it makes me feel that I am not missing anything too big. lol

I am in the process of bringing up a new switched access layer for a client who has purchased 4948s. The 4948s run at wire speed with a 96 Gbps backplane. I am trying to convince them that utilizing only 2 of the 4 SFP uplinks is a waste. Using 2 will give them a 24:1 oversubscription ratio. At least with 4 ports, they will have a 12:1 OS ratio. Both OS ratios are assuming that they use both uplinks for alternating vlans, as we discussed earlier. Otherwise, with one active uplink, itll be 48:1 or 24:1 in the latter case.

Any particular reason they only want to use 2 of the 4 ? seems a bit of a funny choice unless the vast majority of traffic is locally switched. Even then it would be a strange choice.

I think this is one of the cases where you could consider a big pipe between the 2 aggregation switches as even though there would be a lot of interswitch traffic (unless you use contexts) this would still allow you to use both uplinks.

Now you could go L3 but we both know in a DC this isn't really practical

Jon

lamav · ‎05-06-2010

I hear you...

No, no reason I could think of -- or they could give -- for not using all 4 uplinks.

Their network is a disaster. They have over 100 vlans configured and they have spanned them across all their access switches -- all 15 of them lol...

What they did was create the vlans on their agg switches, which is in VTP server mode, and then just had all the access layer/client switches build their vlan databases accordingly.

Now they want to build a new environment, but want to fix the existing one first. They will have two parallel networks. The old environment has multiple uplinks with no STP, so all but one is blocked....some switches only have one uplink altogether...all of then are running IOS that is anywhere between 4 and 7 years old, and most switches havent been rebooted in 4 years....

They have a collapsed core that is also doing load balancing with a CSM and they have about 15 eigrp neighbors between them becase they used the vlans for peers..lol

Sigh....

Victor

Leo Laohoo · ‎05-06-2010

Hiya Victor,

Congratulations on your new badge!

I know the simplest and quickest solution to old "network disaster": Set the place on fire. It's difficult if you try to untangle the mess or fix it in the first place. It's quicker if you build a brand new and network particularly if designed correctly by someone who knows what he's doing (other than a so-called "CCIE" who designed to power up a rack of servers using power from a PoE switch).

The biggest hurdle, in my humble experience, is to physically switch users and servers from the old clunker network to the newer network. Thus, your honour, is the reason why my client set his network on fire. The defense rests it's case. Elvis has left the building ...

lamav · ‎05-06-2010

Leo...lol..what can I say? You made me laugh but good, Elvis...

Thanks!

Leo Laohoo · ‎05-06-2010

Don't laugh about the "CCIE" bit. It's true. It happened just last week.