6509 - Oversubscrption vs Performace Mode

[anis_cisco]

Hello,

We are in stage of deploying VSS between two 6509 ( SUP720-10G + WS-X6708-10G ).

We will use 10G ports of SUP for VSL & We will use 8port 10G module for uplinks to distribution switches.

We went through the configuration guides & come across through oversubscription mode or performance mode.

Q1- We understood the difference between both modes but didnt understand in which scenario these modes work.

Does both modes operate in VSS scenario only ? or it comes in scenario where we need high bandwidth from slot (like in Performance mode some interfaces administratively shutdown & there bandwidth shift to the intrefaces that are up for higher output) ?

Q2- I read that " 8 port module is oversubscribed 2:1 " what is the meaning of 2:1 ? Does it mean 2 ports in 1 PortGroup ?

Q3- What is default mode for 6708-10G slot ? oversubscription or performace ?

Regards

[Jon Marshall · Accepted solution]

1)  It is not dependant on VSS so the modes apply to both VSS and standalone 6500 setup.

2) The WS-6708 has one port group only. What it means by oversubscribed is that with a supervisor 720 you can get 40Gbps per slot. But you have a potential 80Gbps if you fully populate the module so -

80Gbps in (in = from end devices connected to the module)

40Gbps out (out = to the switch fabric)

Now it is highly unlikely all ports will be transmitting 10Gbps at once but what performance mode does is shutdown 4 of the ports so you cannot possibly get oversubscription ie.

40Gbps in (because the other 4 ports are shutdown)

40Gbps out

3) I believe it is overusbscription and you need to enable performance mode. From the Q&A -

Q. Is the 8-port 10 Gigabit Ethernet module not oversubscribed if I only use half the ports?

A. Yes, you can use only ports 1, 2, 5, and 6 to provide 40 Gbps local switching. To make it easier for you to configure your network, we have a new software command for you to go into performance mode. The software command

router(config)#[no] hw-module slot x oversubscription

will administratively disable the oversubscribed ports (ports 3, 4, 7, and 8) and put them in "shutdown" state. In this mode, the user cannot do "no shut" on the disabled ports. When user do "show interface" on the disabled ports, the output will show "disabled for performance" to distinguish between normal port shutdown and shutdown for performance.

Jon

[anis_cisco]

Jon,

Thank you so much to give quick explaination for my basic question.

Is there any recomendation by Cisco that we should use oversubscribed mode & use performance mode in some specific situations ?

As i mentioned we will be using 8port 10G module to connect with 3750 (distribution switch). What mode should i configure, the default oversubscribed or change it to performance ?

Regards

[Jon Marshall · Accepted solution]

It is very difficult to say without knowing how much traffic is being passed through this module and only you know that really as it is your network.

If for example you used the entire module for uplinks to other switches connected by 10Gbps then i would lean towards performance mode. But it really depends ie.

if each switch uplink only ever maxed at 5Gpbs then you could connect 8 switches to the module and still get no oversubscription ie. 8 x 5 = 40. But if some of the switches were transmitting over 5Gbps then you could get problems.

With VSS, if you use any of the ports in the VSL link (although you should always use at least one supervisor port in the VSL link at a minimum) then you would definitely not want any oversubscription because the last thing you want are control packets being lost on the VSL link. But again it depends on what the other ports are doing in terms of throughput.

One thing i'm not sure of ithough s whether, if you use one or more of the ports on the 6708 module in the VSL link, you then need to put the module into performance mode. I don't believe you do but i will have a quick trawl through the docs to see it mentions anything.

But apart from that it really is down to how much traffic you are putting through the module. And it is worth bearing in mind that a lot networks are designed with oversubscription rates between the access and distribution layer so oversubscription in itself is not necessarily a bad thing.

Jon

[anis_cisco]

Excellent Jon

5 Points for you

[Jon Marshall · Accepted solution]

Many thanks. From the VSS Q&A -

Q. What makes up the VSS?

A. The VSS is made up of the following:

• Virtual switch members: Cisco Catalyst 6500 Series Switches (up to two switches with initial release) deployed with the Virtual Switching Supervisor 720 10GE

• Virtual switch link (VSL): 10 Gigabit Ethernet connections (up to eight using EtherChannel) between the virtual switch members. The 10 Gigabit Ethernet connections have to be one or more of the following modules:

– Virtual Switching Supervisor 720 10GE: 10 Gigabit Ethernet uplink ports

– WS-X6708-10G module: any 10 Gigabit Ethernet port

– WS-X6716-10G module: only non-oversubscribed ports can be used for VSL (supported with 12.2(33)SXI)

VSLs can carry regular data traffic in addition to the control plane communication between the two virtual switch members. The distance of VSL link is limited only by the chosen 10 Gigabit Ethernet optics. For example, with a X2-10GB-ER optic, VSL can span distance up to 40 km. (See Figure 3.)

Notice that the WS-X6716 can only use non-oversusbscribed ports but there is no similiar restriction for the 6708 module, although to be honest i am not sure why.

This is only for the VSL link though, not when you just use the module in a 6500 VSS pair.

Jon

[anis_cisco]

Alright got it.

Thanks man have a great day

[anis_cisco]

Hello Jon,

I tried to configure oversubscribtion mode into core but i got following error :

BTI_CS01_ENGR_F2(config)#hw-module slot 2 oversubscription

This command is not supported on module 2.

Here are outputs of module from core switch:

BTI_CS01_ENGR_F2#show module

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6848-GE-TX     SAL1732B0EE

  2    8  DCEF2T 8 port 10GE                     WS-X6908-10G       SAL1732B1CV

  5    5  Supervisor Engine 2T 10GE w/ CTS (Acti VS-SUP2T-10G       SAL17194J7J

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  7c69.f61d.9318 to 7c69.f61d.9347   1.0   12.2(18r)S1  15.0(1)SY4   Ok

  2  7c69.f61d.8df8 to 7c69.f61d.8dff   1.2   12.2(50r)SYL 15.0(1)SY4   Ok

  5  a44c.1177.2ca9 to a44c.1177.2cb0   1.5   12.2(50r)SYS 15.0(1)SY4   Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  1  Distributed Forwarding Card WS-F6K-DFC4-A      SAL1731ANX2  1.0    Ok

  2  Distributed Forwarding Card WS-F6K-DFC4-E      SAL1730ADGN  1.2    Ok

  5  Policy Feature Card 4       VS-F6K-PFC4        SAL1731AJB8  2.1    Ok

  5  CPU Daughterboard           VS-F6K-MSFC5       SAL17152KUX  2.0    Ok

Mod  Online Diag Status

---- -------------------

  1  Pass

  2  Pass

  5  Pass

BTI_CS01_ENGR_F2#

I am not able to connect 10G fiber module (on core) with 1G fiber module (on distribution sw).

On Core fiber module is X2-10GB-SR & on distribution i have 1000 Base GLC-SX-MM.

The interface status shows down:

Te2/1        **** Trunk connect notconnect   routed       full    10G 10Gbase-SR

I tried reducing the speed of 10G interface to 1000 but i didnt find speed option under interface. Any idea that can we connect 10G with 1G interface ?

[Jon Marshall · Accepted solution]

Your original post was about a sup720 with a ws-x6708 but you have a sup2T with a ws-x6908. The sup2T provides 80Gbps per slot so and the 6980 has 8 10Gbps ports -

80Gbps in (from connected devices)

80Gbps out (to switch fabric)

so there is no oversubscription ie. this card can run at linerate.

As for the connection, it's been a while since i ordered transceivers for 6500s so you may want to post that question in a separate post but as far as i understand it you need a twingig converter for the ws-x6908 for this to work. There is a twingig converter for the 6500 but it is not used to convert 10Gbps to 1Gbps it is used to support SFP+ 10GE optics.

So as far as i know you cannot run your 10Gbps ports as 1Gbps but like i say you may want to post that as a separate question because i could be wrong.

Jon

[Joseph W. Doherty]

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

Jon you're another early riser?  I see you posted this while I was still typing my reply.  Your answer must be correct as it agrees with mine, or is it mine agrees with yours? 

BTW, to the OP, as Jon has noted, it's unusual, but not impossible, to actually have problems due to a 6708's oversubscription.  To avoid losing packets, another major difference between a 6704 and a 6708, the latter has much more on-card memory for buffering packets.  Often this will avoid dropping packets if there are transient oversubscription bursts, but then you'll see transient latencies.  The feature to deactivate half the ports, avoids this.

However, understanding a 6708's architecture provides another usage option.  The card's 8 ports are divided into groups of 4 which share 20Gbps (duplex) to the fabric.  So, if you know some of your 10g ports are going to be much more busy than others, you can ballance out loading based on how you connect your ports.  For example, as long as the aggregate of 4 ports connected to one of the fabric channels doesn't (usually) exceed 20Gbps (duplex), you can take advantage of using all 4 10g ports without the (possible) impact of oversubscription.  Also BTW, this principle of how ports map to fabric bandwidth can be applied to many other cards for the 6500 series.

[anis_cisco]

Jon,

I appologize in posting wrong SUP information.

Thank you for clarification.

Regards,

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

I tried to configure oversubscribtion mode into core but i got following error :
BTI_CS01_ENGR_F2(config)#hw-module slot 2 oversubscription
 This command is not supported on module 2.
Here are outputs of module from core switch:
BTI_CS01_ENGR_F2#show module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6848-GE-TX     SAL1732B0EE
  2    8  DCEF2T 8 port 10GE                     WS-X6908-10G       SAL1732B1CV
  5    5  Supervisor Engine 2T 10GE w/ CTS (Acti VS-SUP2T-10G       SAL17194J7J

That's because although your OP asked about a 6708, your installed card in a 6908. This card is not oversubscribed.  It supports 80 Gbps with your sup2T.

I am not able to connect 10G fiber module (on core) with 1G fiber module (on distribution sw).
On Core fiber module is X2-10GB-SR & on distribution i have 1000 Base GLC-SX-MM.
The interface status shows down:
Te2/1        **** Trunk connect notconnect   routed       full    10G 10Gbase-SR
I tried reducing the speed of 10G interface to 1000 but i didnt find speed option under interface. Any idea that can we connect 10G with 1G interface ?

Generally, optic ports only run at one "speed", i.e. you cannot mix and match 10g on one end and gig on the other.  (Similarly, you cannot mix different types of optic of the same speed, e.g. a gig SX won't work with a gig ZX.)

[anis_cisco]

Thank you joseph for your replies & valueable findings

Regards,

[Jon Marshall]

Joseph

Well i live in the UK so it's not that early over here. Our answers on 6500 performance are often interchangeable so think it's fair to  just say we agree with each other

Couple of points on this post though - 

Do you mean 6704 and 6708 (not 6504 and 6508) ?

However, understanding a 6508's architecture provides another usage option.  The card's 8 ports are divided into groups of 4 which share 20Gbps (duplex) to the fabric.

Where did you find this info because according to the docs it is one port group for the 6708 -

http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/hardware/Module_Installation/Mod_Install_Guide/02ethern.html#wp1041391

So it wouldn't matter which ports you used as long as it was only four. Having said that it would not be the first time that there was conflicting information about this sort of thing so are you referring to another document or have i misunderstood your point ?

Jon

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Oops, I did indeed mean 6704 and 6708.

Within your reference, notice:

Fabric connection

•Dual switch-fabric connections: –Fabric channel 1: Ports 2, 3, 6, 8 –Fabric channel 2: Ports 1, 4, 5, 7

And immediately below it:

Fabric channel speed

20 Gb/sec

It's also interesting to note which ports are actually disabled when you enable performance mode.