Active/Active x-bar

martynhowsam · ‎06-19-2009

I'm looking into a design at the moment based on Cisco 9513.

Am I right in thinking that to achieve the 192G full duplex bandwidth per slot I need to have 2 Fabric-2 modules and run them active/active ?

If that is the case what happens if one of them fails - presumably the slot bandwidth drops to 96G full duplex but how do the gen3 line cards handle this as they have 8 Asics each capable of 12.8G - do they drop to 6.4G or do 4 go our of service or something else ?

storagemitch · ‎06-22-2009

Yeah, interesting point. I'd guess that each Fabric 2 module has enough bandwidth to cope with the full 96Gb if one fails.

That's how the Fabric 1 modules work - each can handle the full 48Gb on it's own if necessary. IIRC there are unused channels that kick in when the other Fabric module fails.

This is just my 2c - I'd be interested to get a definitive answer if someone knows for sure.

martynhowsam · ‎06-23-2009

If the 'unused channels' have been used though to increase the 48G to 96G then they would no longer be available when one Fabric-2 module fails - if that is the method that has been used - anyone know for sure ?

storagemitch · ‎07-07-2009

Did you get any more data on this? I'm thinking about how to lay out connections on the 8g blades and this is kinda crucial.

Michael Brown · ‎07-07-2009

As I understand it, with the port group over subscription rules in use when configuring the rate modes, you can not configure over 96GB duplex on any line card...so even if you loose a fabric 2 line card, you do not end up with dedicated rate ports running below the configured dedicated rate. If you use the 'no rate-mode oversubcription module x' command, this only comes into play for 'shared' rate mode ports...so if you configure say 12 ports for 8GB shared, they will never all run at 8GB at the same time anyways...so if you lost a fab2 module, you are still not over subscribed on dedicated rate mode ports.

What the fab2 introduces is the ability to run 8GB dedicated mode across 2 ports on the same line card...in different port groups.

The new channels are opened between the XBAR asics (there are 2 on each fabric module) so that you can run 8 (dedicated) up to 1 XBAR, 8 between X-BARs, and then 8 (dedicated) down to another port group on the same line card.

Your line card layout design should not be impacted by using fab2s and the 8GB line cards.

I hope this helps,

Mike

Michael Brown · ‎07-08-2009

Actually...on the 96GB rate line card, you could end up with enough ports configured for dedicated ratemode at 4GB where by if you loose a fab2, they would drop into shared rate mode.

There is no way to design around the bandwidth reduction, since all line cards would see the loss in BW to the fabric.

This can get confusing!

- Mike