Solved: Re: Avoiding oversubscription on WS-X6904-40G-2T.

jmp780718 · ‎01-17-2014

Hi.

I need to guarantee the 10G of bandwidth of couple of interfaces that are members of port channel on Cisco Catalyst 6500. Someone have best practice to do it.

Thank you.

Jon Marshall · ‎01-18-2014

You can ask as many questions as you like

It has actually made me revisit the 6500 architecture and i probably needed the refresher course. One thing i did say that was misleading was talking about shaping on egress because really in terms of oversubscription we are really concerned with ingress traffic ie. traffic coming from devices connected to the linecard.

The switch fabric connection itself is the limiting factor on egress ie. you cannot get more than 80Gbps into that linecard from the switch fabric anyway but you could certainly get more than 80Gbps into the linecard from devices attached to the linecard. However the WS-6904 only supports shaping on egress so that would be no use here.

In terms of policing i have never used that as a way to try and stop oversubscription although the 6500 does support ingress congestion avoidance. Whether your specific module supports it i can't say without doing some more digging as it doesn't specifically mention it although i may have missed it.

I am not sure how practical it would be or how efficient though ie. if you have 6 ports you need to rate limit to no more than 20Gbps then do you just apply it evenly between the ports which may mean some servers do get dropped packets while there is free capacity (in terms of the 20Gbps) left unused. I think you would either need to accept the inefficiences or have a very good idea of the level to set per device and you could see how this would be quite complicated and difficult to manage.

Your original post asked for best practice and i suspect, although i don't work or speak for Cisco, they may well say if you want to be sure of no oversubscription then enable performance mode but i still think in your situation because you only need two ports to run with no oversubscription that is wasting ports you could be using.

There is a person on these forums who is very good at QOS (and also 6500s) so i will see if he can add to this, especially the part about using rate limiting to address the oversubscription problem but again, as far as i can see if you only use 4 ports on one port group i cannot see how this will be oversubscribed.

Jon

View solution in original post

Leo Laohoo · ‎01-17-2014

6904 is a line card for the 6500 with four (4) 10 40 Gbps interface. Coupled with the fact that you are going to be using Sup2T which, potentially, brings the 6500's backplane speed to 80 Gbps.

So my question is this: What oversubscription are you talking about?

Message was edited by: Leo Laohoo Not reading the post properly. :P

jmp780718 · ‎01-17-2014

That´s correct Leo but the card also can enable with operational mode of TenGigabitEthernet on this mode you have 16

TenGigabit port with oversubscription of 2:1, but i need in this mode to guarantee the 10G of bandwidth of couple of interfaces that are members of port channel, do you have any clue?.

Leo Laohoo · ‎01-17-2014

Ok, I read your post wrong.

So you have a four-port card which can support 40 Gbps per port.

I do believe you can enable over-subscription mode on the card. Basically when you enable over-subscription mode, two of the ports will be PERMANENTLY disabled. This ensures you have non-blocking, line rate 80 Gbps transfer rate.

jmp780718 · ‎01-17-2014

Hi Leo.

The card is enable on TenGigabit in this mode i have 16 TenGigabit with oversubscription of 2:1, but i need couple of interfaces 10G bandwidth guarantee.

Leo Laohoo · ‎01-17-2014

The card is enable on TenGigabit in this mode i have 16 TenGigabit with oversubscription of 2:1, but i need couple of interfaces 10G bandwidth guarantee.

Same thing.

Enable over-subscription and this will permanently disable port 2 and port 4.

Stick two FourX adapter into port 1 and port 3 and you are guaranteed 80 Gbps line rate.

Leo Laohoo · ‎01-17-2014

Go here: http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/data_sheet_c78-696623.html

You will notice in Figure 4, ports 2 and port 4 will be disabled when you enable the over-subscription feature.

Jon Marshall · ‎01-17-2014

Use the following link and scroll down to the Port Performance Modes section where it gives you instructions on what to do -

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11-696669.html

Jon

Leo Laohoo · ‎01-17-2014

Nice link, Jon!

jmp780718 · ‎01-17-2014

Thank you Jon and Leo, sorry for the insistence is possible to have TenGigabit and oversubscription mode, and have the 16 port available and also 10G bandwidth to couple of interfaces. The performance mode leaves some ports permanently disable, i need all the ports availables on the card.

Leo Laohoo · ‎01-17-2014

The performance mode leaves some ports permanently disable, i need all the ports availables on the card.

Exactly how many interfaces do you want guaranteed 10 Gbps?

I am not sure if you can do traffic shaping with Sup2T.

jmp780718 · ‎01-17-2014

Hi Leo.

I need at least 2 port on 10g?, there Is a QoS feature that can help me?

Sent from Cisco Technical Support iPad App

Leo Laohoo · ‎01-17-2014

Ok, if you want 2 running at guaranteed 10 Gbps but you also want fourteen ports at 10 Gbps.

I am no authority on this but if you are able to configure bandwidth SHAPING to the fourteen ports, then this is the solution you should be looking for. QoS is not a good solution because trying to fine-tune QoS values is going to be a b1tch.

Sent from Cisco Technical Support WII App

Jon Marshall · ‎01-18-2014

I don't think shaping is going to work here. You have to visualise the switch fabric connections as in and out.

In means traffic coming from devices connected to your WS-X6904 module and going to the switch fabric (assuming the destination port is not on the same module).

Out is traffic coming from other linecards from the switch fabric going to your WS-X6904 and then out to end devices.

Shaping inbound would require you to know what amount of traffic is coming into each port or divide the ports by the total amount of bandwidth and shape per port on that which would mean some ports would be underutilsed whilst others would be shaped and potentially having to drop packets.

Shaping outbound would not work because the limiting factor here is the actual switch fabric connection itself ie. you can only push 80Gbps into the linecard from the switch fabric anyway.

Looking at the document i linked to it looks like the actual groupings are 5 - 12 sharing a 40Gbps connection to the switch fabric and 13 - 20 sharing s 40Gbps connection to the switch fabric. But the principle remains the same as already outlined.

The actual architecture of the linecard is quite complcated because it can support both 10Gbps and 40Gbps port so i;ll have a read today to make sure i haven't steered you down the wrong path but the limiting factor is always going to be the connections to the switch fabric.

Jon

Jon Marshall · ‎01-18-2014

If you absolutely must run all 16 ports then the only way to do this and ensure no oversubscription for two of the ports is to ensure that the remaining six ports in the port group do not exceed 20Gbps.

You cannot really do this with configuration on the 6500, you have to know that the six devices connected to the port group will never exceed 20Gbps between them. If you know this is the case then you could use all ports but that is a very difficult thing to be certain of and if you get it wrong then you impact upon the two ports you want to run with no oversubscription.

Jon