Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

Bronze

router with cef ,and multilayer switch with cef,

Hi everybody

Let say we have to route traffic between two vlans, vlan1 and vlan2.  We have to choose between CEF-enabled router and CEF-enabled multilayer switch. Further assume  the choice should be made solely on performance  ( i.e cost should not be a reason).  Why should one choose multilayer switch over router in our scenario? 


Have a great weekend.

thanks.

2 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

router with cef ,and multilayer switch with cef,

Hi Sarah,

If solely the routing performance is the reason then obviously, the multilayer switch is the clear choice.

A router, even if running CEF, is still forwarding the packets using its CPU. The "CEF" as a term does not describe some kind of hardware forwarding but rather an efficient organization of the routing table and the related Layer2 header rewrite information, optimized for fast lookups. You are most probably aware that CEF consists of two structures - a FIB and an adjacency table. On routers, both these structures are stored in router's RAM. When routing packets, the router's CPU makes lookups in these FIB and adjacency structures but they are, just as any other runtime information, stored in RAM. The total throughput of a software-based router, even if running CEF, is therefore soon limited by the incurred load on the CPU and additional features performed during the packet routing (ACL, inspection, NAT, ...). Even with no additional features activated, a flow of a couple hundreds Mbps can kill a 2800 series router - the CPU is simply not powerful enough to handle so many packets in a particular time interval. Once again, with software-based CEF routers, all lookups are performed by CPU in RAM where the FIB (in the form of an mtrie) is stored.

On multilayer switches, the FIB part of the CEF is downloaded into TCAM. This TCAM is capable of performing extremely fast parallelized lookups without ever bothering the CPU. The former bottleneck created by the CPU is simply gone and nonexistent. That is one of the primary reasons why multilayer switches have very high throughput and low latencies.

Of course, it has to be taken into account that multilayer switches often lack functionality routers have - elaborate deep packet inspection and NAT being some of the most visible features not usually available on multilayer switches. This has also to be taken into account when choosing an appropriate device.

Best regards,

Peter

Cisco Employee

Re: router with cef ,and multilayer switch with cef,

Sarah,

I actually expected you would ask about this information!

The truth is - I do not know for sure, but my idea is as follows: Even if the adjacency is stored in the normal RAM, you do not need CPU to access it. The TCAM is the embodiment of the lookup algorithm - the result from TCAM can be used as a pointer to the RAM where the appropriate adjacency record is stored. Subsequent accessing the RAM can also be done without CPU's attention - think how DMA (Direct Memory Access) works - the common technique used in PCs for quite long a time to allow peripherals to transfer large bulks of data to and from RAM without CPU's attention or help.

Similar technique can be used in multilayer switches: the destination IP address is fed into the TCAM, producing a hash representing the location of the associated adjacency entry in some kind of RAM, either the system RAM or a RAM associated with the switching matrix. The switching matrix then accesses this adjacency entry and produces a new frame accordingly.

Best regards,

Peter

6 REPLIES
Cisco Employee

router with cef ,and multilayer switch with cef,

Hi Sarah,

If solely the routing performance is the reason then obviously, the multilayer switch is the clear choice.

A router, even if running CEF, is still forwarding the packets using its CPU. The "CEF" as a term does not describe some kind of hardware forwarding but rather an efficient organization of the routing table and the related Layer2 header rewrite information, optimized for fast lookups. You are most probably aware that CEF consists of two structures - a FIB and an adjacency table. On routers, both these structures are stored in router's RAM. When routing packets, the router's CPU makes lookups in these FIB and adjacency structures but they are, just as any other runtime information, stored in RAM. The total throughput of a software-based router, even if running CEF, is therefore soon limited by the incurred load on the CPU and additional features performed during the packet routing (ACL, inspection, NAT, ...). Even with no additional features activated, a flow of a couple hundreds Mbps can kill a 2800 series router - the CPU is simply not powerful enough to handle so many packets in a particular time interval. Once again, with software-based CEF routers, all lookups are performed by CPU in RAM where the FIB (in the form of an mtrie) is stored.

On multilayer switches, the FIB part of the CEF is downloaded into TCAM. This TCAM is capable of performing extremely fast parallelized lookups without ever bothering the CPU. The former bottleneck created by the CPU is simply gone and nonexistent. That is one of the primary reasons why multilayer switches have very high throughput and low latencies.

Of course, it has to be taken into account that multilayer switches often lack functionality routers have - elaborate deep packet inspection and NAT being some of the most visible features not usually available on multilayer switches. This has also to be taken into account when choosing an appropriate device.

Best regards,

Peter

Bronze

router with cef ,and multilayer switch with cef,

Hi Peter.

The  FIB part is loaded into CAM, what about the adjacency table?  Because if adjacency table is not loaded into cam, a switch must  use the ram and thus  cpu to find the next-hop 's mac address. Is that correct?

thanks and have a nice weekend.

Super Bronze

Re: router with cef ,and multilayer switch with cef,

Disclaimer

The    Author of this posting offers the information contained within this    posting without consideration and with the reader's understanding that    there's no implied or expressed suitability or fitness for any  purpose.   Information provided is for informational purposes only and  should not   be construed as rendering professional advice of any kind.  Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In    no event shall Author be liable for any damages whatsoever  (including,   without limitation, damages for loss of use, data or  profit) arising  out  of the use or inability to use the posting's  information even if  Author  has been advised of the possibility of such  damage.

Posting

A good question, that I don't know the truly accurate answer too, but I would suspect MAC's would be handled as normal.  Consider almost all the routing entries are going to use few MACs, as you note, the next hop MAC for particular destination routes.  This normally isn't much of an issue for software based routers either, i.e. converting next hop IP to next hop MAC.

PS:

Returning to your original question, of which to pick, L3 switch or router, strictly on performance, I agree with Peter for the reasons he describes although do keep in mind in Cisco's product line-up, not every "router" is performance limited by is main CPU, like ISRs.  Examples would include the 7200 with NSE-1, the 7304 with NSE-100/150, the 7600 series (of course a rebadged 6500), the CRS series, the ASR 100 series, the ASR 9000 series and the XR 1200 series.  These series "route" in hardware, either (normally) completely or partially.  So, if cost isn't a factor, a CRS-3 might be a good choice

As Peter also described, L3 switches (and hardware based routers) often have fewer feature than a CPU based router.  This because it's both difficult and expensive to implement every feature in hardware.  Normally hardware based routers and L3 switches only deal well with typical traffic; they can perform surprisingly poorly when hit with unexpected, i.e. non-hardware accelerated, traffic.

Years ago, I analyzed why Code Red "melted down" some core "routers".  These were Brand X, but their architecture and perforamance were like a flow-based, not CEF-based, 6500 with sup2 and fabric.  I discovered I could cause one of these to go into meltdown if I forwarded to it about 3 Mbps of packets with each single packet going to different IPs.  Being flow-based, each one of these single packets were software forwarded.  Subsequent packets would have been flow cached in hardware, but there were no subsequence packets for each new flow.

Also years ago, I tested the impact of reducing timing intervals for RIP updates between a pair of 2811 and a 3750G.  When I got down to 1 second updates, the 2811 were doing just fine but the 3750G's CPU was maxed out.  "Control plane" processing on many L3 switches isn't hardware accelerated.  What this seem to demonstrate was the "fast" 3750G's CPU wasn't nearly as fast as the CPU used on the 2811.  (BTW, this is also demonstrates why L3 switches might have CoPP features.)

So in answer to your question about what's the faster platform to route between two VLANs, Peter's reply is 100% correct for "normal" traffic, but if your traffic is "abnormal" (at least as far as hardware acceleration is concerned), some CPU routers could, again in very specific situations, be faster.

New Member

Re: router with cef ,and multilayer switch with cef,

Why do some routers not have TCAMs and instead rely on a software lookup, even for CEF?

Super Bronze

Re: router with cef ,and multilayer switch with cef,

Disclaimer

The     Author of this posting offers the information contained within this     posting without consideration and with the reader's understanding  that    there's no implied or expressed suitability or fitness for any   purpose.   Information provided is for informational purposes only and   should not   be construed as rendering professional advice of any kind.   Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In     no event shall Author be liable for any damages whatsoever   (including,   without limitation, damages for loss of use, data or   profit) arising  out  of the use or inability to use the posting's   information even if  Author  has been advised of the possibility of  such  damage.

Posting

ex-engineer wrote:

Why do some routers not have TCAMs and instead rely on a software lookup, even for CEF?

Allows them to be built less expensively and also allows them to be extremely flexible in what they do.  Remember some pre-CEF routers become CEF routers with just an IOS upgrade.

Cisco Employee

Re: router with cef ,and multilayer switch with cef,

Sarah,

I actually expected you would ask about this information!

The truth is - I do not know for sure, but my idea is as follows: Even if the adjacency is stored in the normal RAM, you do not need CPU to access it. The TCAM is the embodiment of the lookup algorithm - the result from TCAM can be used as a pointer to the RAM where the appropriate adjacency record is stored. Subsequent accessing the RAM can also be done without CPU's attention - think how DMA (Direct Memory Access) works - the common technique used in PCs for quite long a time to allow peripherals to transfer large bulks of data to and from RAM without CPU's attention or help.

Similar technique can be used in multilayer switches: the destination IP address is fed into the TCAM, producing a hash representing the location of the associated adjacency entry in some kind of RAM, either the system RAM or a RAM associated with the switching matrix. The switching matrix then accesses this adjacency entry and produces a new frame accordingly.

Best regards,

Peter

1776
Views
0
Helpful
6
Replies
CreatePlease to create content