Re: CEF vs. EIGRP for load balancing

jason.linden · ‎03-26-2002

Some of our sites have dual PVCs pointing to two different FR T1's on our head-end router, the PVC's have the same port speed, CIR, and the bandwidth command is set to the same for both PVC's, so from EIGRP the paths are equal. My question is how does EIGRP and CEF differentiate in terms of load balancing on a per-packet basis?

I have enabled per-packet on one of the sub-interfaces here is the output for 'sh ip cef detail' for that destination network:

10.0.1.0/24, version 358, per-packet sharing

0 packets, 0 bytes

via 10.75.3.10, Serial0/0.2, 0 dependencies

traffic share 1, current path

next hop 10.75.3.10, Serial0/0.2

valid adjacency

via 10.75.4.10, Serial0/1.2, 0 dependencies

traffic share 1

next hop 10.75.4.10, Serial0/1.2

valid adjacency

25500 packets, 17030273 bytes switched through the prefix

30 second output rate 78 Kbits/sec

My question is why does Serial0/0.2 say 'traffic share 1, current path' but for s0/1.2 it says 'traffic share 1'? What does the current path mean? Does it mean that traffic is only going out the S0/0.2 interface?

One last question... Does CEF need to be enabled on both ends of the Frame to work efficiently?

nuno.morais · ‎03-27-2002

Hi.

You should configure "ip load-sharing per-packet" on both interfaces.

CEF is local to the router, the other end of the link doesn't need to have CEF in order to load balance traffic that is going out of your router.

Rgds.

NM

jason.linden · ‎03-27-2002

Thanks. 'ip load-sharing per-packet' is configured on both interfaces. I was also wondering... Are there any MIB's or another way to extract the '30 second output rate'? I would like to use this to do bandwidth monitoring with trying to calculate the input and output rate the different serial interfaces. Thanks!!

-jason

bhedlund · ‎03-27-2002

http://www.mrtg.org

shengming · ‎03-27-2002

For the interactive transactions, the data are not only transferred out from local, but also received from remote - the remote router also need to config CEF to ensure real balance.

The per-packet sharing will increase the CPU load to reassemble the packets from different routes.

But I don't know what's the situation when one line is more busy than another line, what's the detail scenario for the packet transferring.

Anyone has more information on it?

ruwhite · ‎03-29-2002

Current path means this is the path which will be used for the next packet transmitted to this destination. Of course, it should change as packets are actually switched. :-)

You probably won't ever get 100% equal amounts of traffic on the two links, just because of different packet sizes and statistical discrepancies. I've never seen it work that way, anyway.... CEF accounting might help you see what the traffic levels are better, as well as the interface level counters.

Finally, none of the routing protocols really determine load sharing, pretty much. Think of the rib as a meeting place for the switching path and the routing protocols. The rp's install stuff there that the switching code uses to determine how to switch packets. The routing protocols may install one or more paths to a given destination, with thei associated metrics. Once this is done, the rib figures out, from the metrics, what the ratio of packets between the two links should be. CEF then takes this information and fills in several tables based on it, and switches the packets from these tables.

I think that Inside Cisco IOS Software has some explanation of this, but I don't remember how much. The slides fro PS-540 at networkers should have some explanation as well, along with the slides from the two hour arch session there, if you can get your hands on them.

Russ

shengming · ‎03-31-2002

That sound good, but if there are several hops between destination and source terminal, the lines adjacent to the source router is busy or not does not mean the next lines adjacent to next hops will be busy or not. So the rib calculation is based on source-destination pairs or just the physical lines? do all these routers need to use CEF or just some routers need? thanks!

jason.linden · ‎04-01-2002

That is what I thought the current path meant... However on all of the interfaces that I have CEF per-packet enabled on it always shows that the serial 0/0.x is the 'current path'. Here is a copy of what the config looks like

interface Serial0/0.4 point-to-point

bandwidth 128

ip address XXX.75.3.17 255.255.255.252

no ip directed-broadcast

ip load-sharing per-packet

frame-relay interface-dlci 43 IETF

end

RTRAKR01#sh run int s0/1.4

Building configuration...

Current configuration:

!

interface Serial0/1.4 point-to-point

bandwidth 128

ip address XXX.75.4.17 255.255.255.252

no ip directed-broadcast

ip load-sharing per-packet

frame-relay interface-dlci 43 IETF

end

The 'sh ip cef' output for that remote subnet looks like this:

XXX.75.141.0/24, version 509, per-packet sharing

17806 packets, 5134468 bytes

via XXX.75.3.18, Serial0/0.4, 0 dependencies

traffic share 1, current path

next hop XXX.75.3.18, Serial0/0.4

valid adjacency

via XXX.75.4.18, Serial0/1.4, 0 dependencies

traffic share 1

next hop XXX.75.4.18, Serial0/1.4

valid adjacency

17806 packets, 5134468 bytes switched through the prefix

30 second output rate 32 Kbits/sec

Thanks!!

-jason

ruwhite · ‎04-01-2002

I don't know that this counter is telling you what the real current path is. :-) Have you tried turning on cef accounting to see what it says?

Russ

ruwhite · ‎04-01-2002

I've thought about my last response a bit more, and have another suggestion for you. :-)

What interface is this traffic inbound on? What sort of a box is this? Try enabling per packet on the inbound interface.

:-)

Russ

shengming · ‎04-01-2002

Russ, what will be the best configuration for load balance if there are several hops between destination and source, also several routes. for example from S to D: S connect to A, then to D; S also connect to B, then to D; A and B have another valid connection. What's the best way to set the load balance between S and D if all the application systems are located in D, users in S/A/B are using these applications through terminal servers? Maybe we can think the problem in another direction. Thanks!

ruwhite · ‎04-02-2002

It depends on the number of sources and destinations--if there are a large'ish number of both, then you should be fine with per source/destination pair. Watch for issues with load sharing polarization when using this in earlier code, however.

Per packet is fine as long as either out of order packets won't affect the application, or the links are sync'd closely enough to cause very few or no out of order packets.

Russ

jason.linden · ‎04-02-2002

Here is what the traffic flow looks like:

22.0 -> fa0/0 -> s0/0.4 or s0/1.4 -FR- -> s0/0.1 or s0/0.2 -> fa0/0 -> 141.0

141.0 is the remote LAN subnet, 22.0 is our data center subnet. Right now 'ip load-sharing per-packet' is enabled on all serial interfaces. Our data center router is a 3640 running 12.0(7) and the remote site is a 2620 also running 12.0(7). However over the course of the next month I am going to be upgrading all of our routers to 12.2(somthing). Is CEF more mature in 12.2?

So you are suggesting that I turn on ip load-sharing per-packet on the fa0/0 interface on our data center router since the packets are coming in off of that subnet?

I have ip cef accounting per-prefix turned on. Could you also explain what the difference is in:

per-prefix, prefix-length, and non-recursive.

I am still searching the internet for the networkers Powerpoint presentation, I am going to Networkers this year but that is not until June!

Thanks again!!

-jason

ruwhite · ‎04-02-2002

Okay--to attempt ascii:

/--s0/0.4--\ /--s0/0.1--\

22.0-fa0/0--rtr1 rtr2 rtr3-141.0

\--s0/1.4--/ \--s0/0.2--/

Is this right? And where are you looking at the cef information? rtr1, or rtr2? And where is per packet turned on? On the e/net, or just the serials?

Last year's n/w slides should be on Cisco online, someplace.... This year's will be better (I just redid them last week, and we're still working on various parts of the different presentations.).

Russ

jason.linden · ‎04-02-2002

Actually, there is no 'rtr2' If you would replace rtr2 with a frame cloud then that is how it looks like. rtr1 and rtr3 are 'directly connected'.

I am looking at the CEF info from rtr1. Per packet is just turned on only the serial interfaces.

I did find the last years presentation after digging more, for those of you who would like to see them: http://www.cisco.com/networkers/nw01/pres/preso/

Thanks!