Generation 3 oversubscription - anyone know how it works?

[bfeeny]

I am trying to figure out how Gen3 modules handle oversubscription.  With the 1st and 2nd gen modules it was very simple.  12.8Gbps per port, and you could oversubscribe to a certain ratio on a given module.

The gen3 documentation says the modules have up to a 10:1 oversubscription ratio.

However, it seems Gen 3 does it differently.  You can look at a table 15-20 in the CLI guide under section entitled "Configuring Oversubscription Ratio Restrictions".  You will see it tells you how much reserved bandwidth is required to run a port in various modules at various speeds.  For the 48 port module you need .24GB of bandwidth per port configured for 4GB.  This means we need .96GB available, but we only have .8GB available.  This is why we have to shutoff ports.  If we shutoff 2 ports (.96 - . 48 ) that gives us .48GB needed.

What I am confused about is why do I only need .24GB for a 4GB port?  That's not 10:1, that's more like 16:1.  Can anyone make sense of the table I reference and break down how to understand Gen3 oversubscription?

Brian

[storagemitch]

Hi Brian

I didn't think there was any real difference with the Gen 3 modules. You still get 12.8Gb per port group to play with, it's just that the port groups are smaller.

On the 48-port blade it means 12.8Gb to share between 6 ports. One port dedicated to 8Gb, another to 4Gb leaves 0.8Gb for the other 4 ports. A 4Gb shared mode port needs a minumim of .24Gb to come online so you can only bring up 3 of the remaining 4 ports (3 x 0.24 = 0.72Gb) but they are highly over-subscribed (0.8/3 = 0.27Gb per port = 15:1 @ 4Gb). All that over-subscribed means is 'the port says 4Gb but there isn't the bandwidth to cash that check'.

Yes, it sucks. What is advertised as a 48-port 8Gb module really gives:

     8 ports @ 8Gb

     8 ports @ 4Gb

     24 ports @ 0.27Gb (no real use in 2010)

     8 ports in power-saving mode (disabled)

Whatever happened to the 256Gb/slot MDS we were promised for 32 x 8Gb line rate ports per module?

Mitch