Solved: How does OSPF load balance on equal cost links w/ CEF?

johnnylingo · ‎08-03-2007

Very basic question, but I don't know how to answer this. Say an access router is running OSPF and has adjacencies to 2 Layer 3 switches. Both links are Gigibit and equal cost. I see in the routing table that all routes to one of the Layer 3 switches. Why is this?

Config:

!

ip cef

!

interface Loopback0

ip address 10.2.22.10 255.255.255.255

!

interface GigabitEthernet0/0

ip address 10.2.2.10 255.255.255.0

ip pim dr-priority 0

ip pim sparse-dense-mode

ip ospf priority 0

duplex auto

speed auto

media-type rj45

!

interface GigabitEthernet0/1

ip address 10.2.3.10 255.255.255.0

ip pim dr-priority 0

ip pim sparse-dense-mode

ip ospf priority 0

duplex auto

speed auto

media-type rj45

!

router ospf 100

log-adjacency-changes

redistribute connected subnets

network 10.2.2.0 0.0.0.255 area 0

network 10.2.3.0 0.0.0.255 area 0

network 10.2.22.0 0.0.0.255 area 0

!

sh ip ospf nei:

Neighbor ID Pri State Dead Time Address Interface

10.2.244.3 1 FULL/DR 00:00:38 10.2.3.1 GigabitEthernet0/1

10.2.255.2 1 FULL/DR 00:00:34 10.2.2.1 GigabitEthernet0/0

What really confuses me is the route table shows all routes going out Gig0/1, but if I look at a specific route:

#sh ip route 10.2.250.0

Routing entry for 10.2.250.0/24

Known via "ospf 1", distance 110, metric 2, type intra area

Last update from 10.2.2.1 on GigabitEthernet0/0, 00:13:58 ago

Routing Descriptor Blocks:

10.21.3.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/1

Route metric is 2, traffic share count is 1

* 10.2.2.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/0

Route metric is 2, traffic share count is 1

I though * meant that this was the preferred route...

AJAZ NAWAZ · ‎08-04-2007

" Routing entry for 10.2.250.0/24

Known via "ospf 1", distance 110, metric 2, type intra area

Last update from 10.2.2.1 on GigabitEthernet0/0, 00:13:58 ago

Routing Descriptor Blocks:

10.21.3.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/1

Route metric is 2, traffic share count is 1

* 10.2.2.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/0

Route metric is 2, traffic share count is 1

I though * meant that this was the preferred route... "

You are correct in that the '*' indicates the preferred route. Issue a ping or send some other traffic again from the access layer up the Gig links. Then issue the same command:

#sh ip route 10.2.250.0

You should see the '*' toggle between the two links. The reason for this is CEF uses per session load-balancing and not per-packet load-balancing. This type of LB is sometimes also known as per-destination LB.

<http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s21/pplb.htm#wp1027122>

'Per-Destination and Per-Packet

Per-destination load balancing allows the router to distribute packets based on the destination address, and uses multiple paths to achieve load sharing. Packets for a given source-destination host pair are guaranteed to take the same path, even if multiple paths are available. For example, given two paths to the same network, all packets for destination1 on that network go over the first path, all packets for destination2 on that network go over the second path, and so on. Per-destination load balancing is enabled by default when you start the router, and is the preferred load balancing for most situations.

Per-packet load balancing allows the router to send successive data packets over paths without regard to individual hosts or user sessions. It uses the round-robin method to determine which path each packet takes to the destination. With per-packet load balancing enabled, the router sends one packet for destination1 over the first path, the second packet for (the same) destination1 over the second path, and so on. Per-packet load balancing ensures balancing over multiple links.

Although path utilization with per-packet load balancing is beneficial, packets for a given pair of source-destination hosts might take different paths. This means that per-packet load balancing can introduce reordering of packets. This load balancing method would be inappropriate for certain types of data traffic (such as voice traffic over IP) that depend on packets arriving at the destination in sequence.

Use per-packet load balancing to ensure that a path for a single source-destination pair does not get overloaded. If the bulk of data passing through parallel links is for a single pair, per-destination load balancing overloads a single link while other links have very little traffic. Enabling per-packet load balancing allows you to use alternate paths to the same busy destination. '

please rate the post if it helps.

hth,

Ajaz

View solution in original post

AJAZ NAWAZ · ‎08-04-2007

" Routing entry for 10.2.250.0/24

Known via "ospf 1", distance 110, metric 2, type intra area

Last update from 10.2.2.1 on GigabitEthernet0/0, 00:13:58 ago

Routing Descriptor Blocks:

10.21.3.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/1

Route metric is 2, traffic share count is 1

* 10.2.2.1, from 10.2.251.3, 00:13:58 ago, via GigabitEthernet0/0

Route metric is 2, traffic share count is 1

I though * meant that this was the preferred route... "

You are correct in that the '*' indicates the preferred route. Issue a ping or send some other traffic again from the access layer up the Gig links. Then issue the same command:

#sh ip route 10.2.250.0

You should see the '*' toggle between the two links. The reason for this is CEF uses per session load-balancing and not per-packet load-balancing. This type of LB is sometimes also known as per-destination LB.

<http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s21/pplb.htm#wp1027122>

'Per-Destination and Per-Packet

Per-destination load balancing allows the router to distribute packets based on the destination address, and uses multiple paths to achieve load sharing. Packets for a given source-destination host pair are guaranteed to take the same path, even if multiple paths are available. For example, given two paths to the same network, all packets for destination1 on that network go over the first path, all packets for destination2 on that network go over the second path, and so on. Per-destination load balancing is enabled by default when you start the router, and is the preferred load balancing for most situations.

Per-packet load balancing allows the router to send successive data packets over paths without regard to individual hosts or user sessions. It uses the round-robin method to determine which path each packet takes to the destination. With per-packet load balancing enabled, the router sends one packet for destination1 over the first path, the second packet for (the same) destination1 over the second path, and so on. Per-packet load balancing ensures balancing over multiple links.

Although path utilization with per-packet load balancing is beneficial, packets for a given pair of source-destination hosts might take different paths. This means that per-packet load balancing can introduce reordering of packets. This load balancing method would be inappropriate for certain types of data traffic (such as voice traffic over IP) that depend on packets arriving at the destination in sequence.

Use per-packet load balancing to ensure that a path for a single source-destination pair does not get overloaded. If the bulk of data passing through parallel links is for a single pair, per-destination load balancing overloads a single link while other links have very little traffic. Enabling per-packet load balancing allows you to use alternate paths to the same busy destination. '

please rate the post if it helps.

hth,

Ajaz

prima.ramadhan · ‎01-28-2010

Hi Ajaz,

I noticed this problem on my router. I use 3 links routed using OSPF, but the utilization is not the same, moreover the difference is very significant. I have one link utilized upto 650 Mbps, while the other two are just 330 Mbps and 130 Mbps. Do you know how to make them more balanced? I am using Cisco 7609 running IOS SRD2a on ES20-GE-3CXL. Actually I tried to change the load balance setting to per-packet, but unfortunately, the command is not there. I also wanted to try to not using CEF, but I remembered that it is impossible as I am running MPLS on my network. Is there any solution to my problem?

Regards,

Prima

AJAZ NAWAZ · ‎08-04-2007

Hey Johnny,

IMO this is a very good document that makes it absolutely clear for us:

<http://www.cisco.com/warp/public/105/46.html>

Ajaz :)