Router 7609 architecture

Unanswered Question
Apr 22nd, 2008

Hi

I'll appreciate if you could help me , I have MPLS network , router 7609 connected to voice getaway through Ethernet module 6748,6548 Some time the voice quality not good

When I check the router used commend (show interface) no error or packet drop

Would like to ask you about:

1) we need to know the internal architecture for Module including the B.W rates for internal and external communication with other modules, what are the bottle necks points that we need to consider while planning?

2) For the show commands like show fabric utilization, and show counters interface x/y, we need to know what are the MIB values that these values query so that we can add them to the monitoring system, and if there are any show commands that can display packet drops in any place on the router whether it is a port or bus or backplane or buffer,…




  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (5 ratings)
Loading.
Jon Marshall Wed, 04/23/2008 - 22:55

WS-X6748-GE-TX has 2 x 20Gbps connection to switch fabric so once you go over 40 ports you may face some contention. Unlikely though as you would have to be pushing serious amount of data.


WS-X6548-GE-TX is a totally different story. It has an 8Gbps connection to switch fabric but it gets worse. Each group of 8 ports share an ASIC and the ASIC provides 1Gbps connection to switch fabric so you can have a serious amount of contention on these modules. If you suspect one of your modules i would start with this one.


Have you implemented QOS on these modules ?


Jon

mohammedmahmoud Wed, 04/23/2008 - 23:35

Hi,


I am sorry, please allow me to interrupt this thread. Jon, what about WS-X6416-GBIC, i didn't succeed in finding any document stating its case explicitly, but from inspecting the card structure, i can see that every 4 ports share the same circuits (and thus i assumed that they share the same ASIC).


BR,

Mohammed Mahmoud.

Jon Marshall Thu, 04/24/2008 - 00:05

Hi Mohammed


Hope things are going well.


The WS-X6416-GBIC is a classic line card so it connects to the 32Gbps shared bus.


The port groupings shown in the release notes don't always tie up. It's been something i've been meaning to investigate for a while. For instance the WS-X6548-GE-TX has 2 port groupings. With an 8Gbps connection to the switch fabric that should mean every 6 ports gets 1Gbps.


But i believe that the WS-X6548-GE-TX has groupings of 8 ports per pinnacle ASIC and this ASIC connects to the switch fabric at 1Gps speed but this obviously is different than the port groupings shown in the release notes, hence they seem to be talking about different things.


According to the release notes the WS-X6416-GBIC has 2 port groupings, ports 1 - 8 and ports 9 -16.


Not sure i have answered your question to be honest but it's the best i can do :-)


Jon

mohammedmahmoud Thu, 04/24/2008 - 01:01

Hi Jon,


Thank you very much, i am doing well, hope u too, and i really hope that you've progressed in your CCIE studies :)


As for the WS-X6416-GBIC, what you described is exactly what confused me, its a classical line card, and thus it is connected to the 32Gbps shared bus, being grouped into 2 groups, each using a single pinnacle ASIC which connects to the switch fabric with 1 Gbps is very odd, it seems that i was too optimistic to assume that it is grouped into 4 groups :) now, this means that every 8 ports shares 1G which is very limited, do you confirm this ?


BR,

Mohammed Mahmoud.



Jon Marshall Thu, 04/24/2008 - 02:05

Mohammed


I think thats where my confusion comes from. For example the 6748-GE-TX has 4 ports groupings but this doesn't mean that each group only gets 1Gbps between them. For the 6748 2 groups of ports get 20Gbps between them. So there is only a slight oversubscription.


I'm not sure how the port groupings in the release notes relate to the internal structure of the module. Perhaps someone from Cisco can share some documents ?


Jon

mohammedmahmoud Thu, 04/24/2008 - 03:21

Hi Jon,


Totally agree with you, its a real mess, i will either try to get a confirmation from Cisco, or i'll try testing it my self.


Thank you very much for your precious time.


BR,

Mohammed Mahmoud.

Jon Marshall Thu, 04/24/2008 - 03:30

Mohammed


No need to thank me, you've helped me any number of times. Thanks for rating.


I will also look into this and let you know if i find any more out.


Jon

t_mostafa Sun, 05/04/2008 - 12:44

Hi jon,


Thanks for your fast response.I checked the gateway & found packet drop.But for the 7609 ,no Packet drop was found.

I would like to have more details about already mentioned topic & problem.Could you please send me documents that could help me have more details.


Thanks on Advance.

Regards,

Tamer

mounir.mohamed Mon, 05/05/2008 - 00:59

Dear Tamer,


As you know Cisco designed 7600 series mostly very near to Cisco 6500 series architecture but with some intelligent features.


Cisco 7600 (As well as Cisco 6500 with SFM module or SP720) has 2 backplanes Classic-Bus or Shared bus (32Gbps) and Cross-bar Switching fabric, the Supervisor and line cards control which BP are gone used.


The common used Processors for 7600 are SP-32/SP-720 and RSP 720, SP-32 support up to 32Gbps centralized processing and connected to Classic-Bus backplane only, so SP-32 is the best option for line cards connected to Classical-bus only (Called non-fabric enabled on 6500 documents)


SP-720 and RSP-720 support up to 720 Gigabit non-blocking bandwidth distributed processing (with line cards have DFC option)

(RSP720 has the same design of SP 720 but with new MSFC and extra memory option)



With a Supervisor Engine 720, traffic is forwarded to and from modules in one of these modes:


-Compact mode-the router uses this mode for all traffic when only fabric-enabled modules are installed in this mode.


-Truncated mode-the router uses this mode for traffic between fabric-enabled modules when there are both fabric-enabled and nonfabric-enabled modules installed


-Bus mode-the router uses this mode for traffic between nonfabric-enabled modules and for traffic between a nonfabric-enabled module and a fabric-enabled module. In this mode, all traffic passes between the local bus and the supervisor engine bus


Line cards classifed like this:


1-Fabric-Enabled line cards connected to Dual 20-Gbps (40 Gbps total) channel connection to switch fabric on Supervisor Engine 720


2-Non-Fabric Line card connected to 32-Gbps Classic-bus (all supervisors)


So Non-fabric line cards (like WS-X6548-GE-TX and WS-X6148-GE-TX) have connection to Classic-bus or 32 Shared-bus but only 8Gbps provided so this card can not use more than 8Gbps of the Classic-Bus backplane


Regarding the line cards Oversubscription issue, since the line cards have 48 ports while it's connected to the backplane by only one 8Gbps connection there is must be Oversubscription model on the line card itself to enhance the uplink utilization which is 8Gbps


So these models of line cards control the uplink bandwidth by share single 1-Gigabit Ethernet uplink from the port ASIC that supports eight ports (Each 8 ports share the same ASIC) and since each block of eight ports is 8:1 oversubscribed. The aggregate throughput of each block of eight ports cannot exceed 1 Gbps.


So you can potentially see slow response due to packet loss if one port of such 8 ports utilized the while ASIC bandwidth.


http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00801751d7.shtml#ASIC


Hope that help


Best Regards,

Mounir Mohamed

guruprasadr Mon, 05/05/2008 - 01:26

Hello Experts,


I have a Clarification with all these 7206 / 7609 Series Routers.


The Question is:


Normally, you can see equal number of SLOTS available ie., the left pane & right pane to load the Cards.


Is there any recommendation from CISCO / any Document to say, whether the card's are supposed to be equally loaded on either side of the pane (left & right)


Example:

========

Consider i have to install 4" Cards (Eg:PA-T3 Serial). Should i install 2" on Left side & 2" on right side (or) can i install all the 4" cards on one end of the Router itself ?


If loading of all 4" Cards are done at one side means, will it somehow impact the performance of the Router.


Please provide your recommendations. Thanks in Advance.


Best Regards,


Guru Prasad R

mounir.mohamed Mon, 05/05/2008 - 01:40

Hi guruprasadr,


On mid-range devices like 7200 Cisco control that by something called bandwidth of points, the box have bus 0 and bus 1, bus 0 control the left slots and bus 1 control slots on the right, each PA utilized specific amount of points (for example PA-A3-OC3 will consume 300 points) so you can not install more than 2 PA-A3-OC3 on the same bus.


This fact changed in legacy 7500 series, which is come with 2 shared bus called Cybus (Cisco Extended bus) each bus support up to 1.067Gbps, and each Bus control/serve number of slots.


In High end devices like 7600 Supervisor and line cards types control this, because each line card limited by the amount of bandwidth allocated to the backplane uplink which is connect the line card to Classic-bus or Switch-Fabric, read the above post fore line cards connections.


So you are free to mix the line cards or port-adaptors as soon as this Comply to the system architecture.


Please rate helpful posts.


Best Regards,

Mounir Mohamed

Jon Marshall Mon, 05/05/2008 - 03:10

Mounir


Excellent post but just one thing if you don't mind.


The WS-X6548-GE-TX is fabric enabled and has a dedicated 8Gbps connection to the switch fabric. If you have a Supervisor 2 in the chassis then you need an SFM (Switch Fabric Module) in slot 5/6 (of a 6509) to provide connections to the switch fabric for fabric-enabled line cards.


Without an SFM then yes it will just use the classic 32Gbps bus. With a sup720 the SFM is built into the supervisor so with a sup720 the WS-X6548-GE-TX would get an 8Gps dedicated connection to the switch fabric.


Jon

mounir.mohamed Mon, 05/05/2008 - 03:24

Dear Jon,


Sure, you are more than welcomed :)


I think I got your point, but please correct me if I'm wrong.


As far as i know non-fabric line cards can only connected to 32Gbps classic-bus.


Fabric-enabled line cards can be connected to either classic and Switch Fabric (SFM for 6500 series / built in on SP-720 and RSP-720)


BUT Fabric-Only cards can be connected to the Switch Fabric ONLY.


But do you mean that WS-X6548-GE-TX (which is Fabric-Enabled) will get only 8Gbps of 40Gbps even if SFM or SP/RSP720 begins used? and this Is because the PINNACLE ASICs of the line card itself already limit the total ports throughput to 8Gbps?


Best Regards,

Mounir Mohamed

guruprasadr Mon, 05/05/2008 - 03:30

HI Mounir,


Thanks for your POST and that was really helpful.


So, you mean to say based on the support of the SYSTEM Architecture designed we can load the Cards ?


I am really happy that i found this POST to discuss about the Topic that was bugging me for long-time.


Kindly clarify is there any CISCO Documents available to study in details about the SYSTEM Architecture (or) any Tools available like "Error Message Decoder" (as in Tools Section) to analyse the capability / support of the Router before planning to add a CARD in BUS 0 / 1 ?


Kindly provide your recommendations.


Thanks in Advance.


Best Regards,


Guru Prasad R

mounir.mohamed Mon, 05/05/2008 - 03:45

Hi guruprasadr,


Welcome man.


For Cisco 7200 Series Bandwidth of points:

http://www.cisco.com/en/US/prod/collateral/routers/ps341/prod_presentation09186a008009184d.pdf


Show Ver on Cisco 7200 will show you the current consumed bandwidth of points on each bus.


For Cisco 7500 Series Cbus there is a full-book in the below URL:

http://www.cisco.com/en/US/products/hw/routers/ps359/products_installation_and_configuration_guide_chapter09186a00801c63a5.html


For Cisco 7600 Architecture and a lot of online PDF's:

http://www.cisco.com/en/US/products/hw/routers/ps368/prod_brochure09186a0080092247.html



Hop that help.

Please rate helpful posts.


Best Regards,

Mounir Mohamed

Jon Marshall Mon, 05/05/2008 - 03:48

Mounir


"But do you mean that WS-X6548-GE-TX (which is Fabric-Enabled) will get only 8Gbps of 40Gbps even if SFM or SP/RSP720 begins used?"


Yes it will. That's really why Cisco position this blade for the wiring closet because used as a server blade it has too much oversubscription.


This blade was one of the early fabric line cards and it only provides 8Gbps to the switch fabric. In theory that means each groups of 6 ports gets 1Gbps each but it is more complicated than that because in reality each group of 8 ports only gets 1Gbps. That's what me & Mohammed were discussing earlier in this thread :-).


Different generations of fabric enabled line cards supports different connections to the switch fabric but as you say one thing all fabric enabled line cards have in common is that they also support a connection to the 32Gbps shared bus.


To summarise


Classic cards can only connect to the classic 32Gbps bus.


Fabric Enabled cards can connect to either the classic bus or the switch fabric dependant on their being either an SFM or a SUP720 in the chassis.


Fabric only line cards can only connect to the switch fabric.


Hope i've answered your question(s)


Jon



mounir.mohamed Mon, 05/05/2008 - 03:54

Sure that's clarified my question


extremely helpful :)


Best Regards,

Mounir Mohamed

Actions

This Discussion