Solved: IP CEF routing vs Local Switching Performance

esdouglas · ‎06-21-2009

I've been questioned many times by my server guys about putting backup servers on the same VLAN as the servers they will be backing up. I argue with IP CEF that the backup servers can be on a separate VLAN and still acheive the same backup speeds as if it were switched locally (same vlan).

What I would like to know is how much delay does IP CEF introduce (if any) vs locally switched traffic. Given that all servers are connected to the same Cat 6509 with Sup720's and 6700 series ethernet modules.

Joseph W. Doherty · ‎06-22-2009

For most backups, even huge ones, net backup times should also be nearly identical. The reason being, any additional per packet forwarding delay just (mainly) shifts packet's arrival time, from sender; it doesn't really impact overall transfer time/rate.

This issue would be somewhat similar to having backup cross one L2 switch or 10 L2 switches. Each switch will add some latency, but assuming overall bandwidth isn't impeded, it will make little difference to overall transfer time.

For bulk transfers, usually the question is whether the platform supports the desired bandwidth transfer rate, not forwarding latency per packet. The latter only becomes important when dealing with latency sensitive apps.

View solution in original post

kishan1984 · ‎06-21-2009

there is no comparison between cef and locally switched traffic,remember cef is switching technology,if u have ur all server connected on same 6509 switch with sup720 so it should 1)Delivering scalable forwarding Performance: up to 400 Mpps1 IPv4 ,2)Delivering up to 40 Gbps per slot of switching capacity; 720 Gbps aggregate bandwidth

esdouglas · ‎06-21-2009

That makes sense. I guess because CEF is enabled on the routed interfaces I seem to think it has to do with routing. I guess if I just remember cisco's mantra I will get it straight...

"switch if you can, route if you must"!

Thanks for your response

Joseph W. Doherty · ‎06-22-2009

For platforms that support CEF in hardware, L3 and L2 forwarding are nearly identical.

Even plaforms that have flow based L3 architectures, e.g. 6500 with sup1, L3 and L2 performance are nearly identical, except for a packet's 1st packet. (Some worms that make individual packets for constantly changing addresses will tank a flow based architecture.)

Even with CEF, there are conditions where some L3 packets will be software routed. Normally, doesn't happen, but if it does, performance will tank.

The old mantra, "switch if you can, route if you must", is still true when you compare software based routers (e.g. ISRs, 7200s) vs. L2 switches, but doesn't really much apply with L2 switches vs. L3 switches (except as noted above).

esdouglas · ‎06-22-2009

Thanks for the response.

The one thing that I keyed on in your response is the "nearly identical" statement, which is probably fair to say for regular user application traffic. But for a backup job that sends millions of packets over its duration, is the difference in delay, multiplied by the number of packets enough to increase the backup duration significantly (30 mins, 1hr??). What I trying to ask is, is the difference <1ms, 5ms, 10ms, etc... I know the best way to answer this question is to run a packet capture, but I'm hoping the answer is readily available somewhere on Cisco's site, or someone's own personal testing.

Joseph W. Doherty · ‎06-22-2009

For most backups, even huge ones, net backup times should also be nearly identical. The reason being, any additional per packet forwarding delay just (mainly) shifts packet's arrival time, from sender; it doesn't really impact overall transfer time/rate.

This issue would be somewhat similar to having backup cross one L2 switch or 10 L2 switches. Each switch will add some latency, but assuming overall bandwidth isn't impeded, it will make little difference to overall transfer time.

For bulk transfers, usually the question is whether the platform supports the desired bandwidth transfer rate, not forwarding latency per packet. The latter only becomes important when dealing with latency sensitive apps.

ronshuster · ‎06-22-2009

Is enabling CEF as simple as running this on the global config:

ip cef

Joseph W. Doherty · ‎06-22-2009

Depends on platform and/or IOS.

On some platforms/IOSs, believe CEF is enabled by default. Some platforms might also not allow it to be disabled, at least globally. (Believe both true for later 6500/IOS.)

BTW, CEF is often required to be enabled to support other features, i.e. something we generally want enabled, if it's supported.

esdouglas · ‎06-22-2009

This answer puts in a much better context for me.

Thanks to all for their responses.