iBGP Multipath Load Balancing on 2 equal cost paths does not work

Unanswered Question
Dec 2nd, 2009
User Badges:

Hi,


My network is as follows -


1. at Data Center i have one 3750 Switch Stack (2 x 3750 Switches in stack) acting as a L3 Core Switch. It connects to 2 WAN routers cisco 2811.


2. at DR i have only one 3750 Switch, no switch stack. This siwtch connects to 2 WAN routers cisco 2811.


3. both the WAN routers at DC & DR connects to each other using Point-to-Point leased lines. ie., DC_R1 connects to DR_R1 and DC_R2 connects to DR_R2


4. DC Core Switch and WAN Routers are configured with BGP AS 65534. DR Core Switch and WAN Routers are configured with BGP AS 65532. DC Core Switch & DC WAN Routers establish a iBGP session, same in DR. Whereas DC WAN Routers & DR WAN Routers establish eBGP session.


5. It is required by customer to load balance and load share traffic between the 2 x WAN routers, so i need to see 2 x routes in the L3 Core Switch at DC & DR. It is configured under BGP with command "maximum-path ibgp 2". This allows the 2 x learned BGP routes into IP Routing table.


6. In DR Core Switch i see 2 equal cost routes and "multipath" in "show ip bgp x.x.x.x", but where as in DC i do not see "multipath" in show ip bgp. Though there are 2 x equal cost paths from the DC core switch, only one best path is shown in show ip bgp.


7. what is difference between the metric just next to the neighbor from which it is learned the route and the metric 0 below it. please see the show ip bgp output


DC_L3_SW#sh ip bgp 164.53.211.198              


BGP routing table entry for 164.53.211.192/27, version 45

Paths: (2 available, best #2, table Default-IP-Routing-Table)

Multipath: iBGP

  Advertised to update-groups:

     2        

  65532

    10.177.44.186 (metric 2172416) from 10.177.43.70 (10.177.44.185)

      Origin IGP, metric 0, localpref 100, valid, internal

  65532

    10.177.44.178 (metric 1764352) from 10.177.43.66 (10.177.44.177)

      Origin IGP, metric 0, localpref 100, valid, internal, best


DR_L3_SW#sh ip bgp 164.53.211.128


BGP routing table entry for 164.53.211.128/27, version 98

Paths: (2 available, best #1, table Default-IP-Routing-Table)

Multipath: iBGP

  Advertised to update-groups:

     2        

  65534

    10.177.44.177 (metric 2172416) from 10.177.43.74 (10.177.44.178)

      Origin IGP, metric 0, localpref 100, valid, internal, multipath, best

  65534

    10.177.44.185 (metric 2172416) from 10.177.43.78 (10.177.44.186)

      Origin IGP, metric 0, localpref 100, valid, internal, multipath


i have verified the cost value (bandwidth) of all the interfaces in path, and they all match. Serial interfaces are configured bandwidth to 2048 kbps on both WAN routers. Fast ethernet links are bandwidth 100mbps.


Please ref to the Network Diagram attached.


Please can any one help on how to solve this problem, DC L3 Core Switch is also required to have 2 x equal cost routes to the DR subnet.


Regards,

Keshava Raju

Attachment: 
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Jon Marshall Wed, 12/02/2009 - 09:36
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Could you provide -


"sh ip route" from the 3750 switches

config of routers


Jon

Laurent Aubert Wed, 12/02/2009 - 09:38
User Badges:
  • Cisco Employee,

Hi keshava Raju,


From your output:



10.177.44.178 (metric 1764352) from 10.177.43.66 (10.177.44.177)

      Origin IGP, metric 0, localpref 100, valid, internal, best


1764352 represent the IGP metric to the BGP next-hop which is 10.177.44.178 in this case.


metric 0 represent the MED associated to the prefix.


It means that DC_L3_SW had computed a different EIGRP metric to reach 10.177.44.178 and this is what you need to fix as iBGP multipath will take place only if the IGP cost to each BGP next-hop are the same.


EIGRP is using a composite metric so you need to be sure bandwidth and delay are exactly the same on every interface.


HTH


Laurent.

Keshava Raju. P Wed, 12/02/2009 - 23:49
User Badges:

Hello Mr. Laurent,


Thanks for your feedback... it really helped.


But the problem still persists.


In India the Point-to-Point leased lines provided by ISP is 2MB (2048 kbps). I have changed the bandwidth on the Serial Interfaces of all the 4 routers to 2048 kb. But still in the routing table, route from the DC_L3 switch to the DR Serial Interface it shows as bandwidth 1544 kb. How can this happen?


Please find the below show ip route & show ip eigrp topology from the DC & DR L3 core switch -


*************************************************************************************************************************************************



DC_L3_SW#sh ip eigrp topology 10.177.44.184/30

EIGRP-IPv4:(999) (AS 999): Topology Default-IP-Routing-Table(0) entry for 10.177.44.184/30

  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 1764352

  Descriptor Blocks:

  10.177.43.70 (Port-channel2), from 10.177.43.70, Send flag is 0x0

      Composite metric is (1764352/1761792), Route is Internal

      Vector metric:

        Minimum bandwidth is 2048 Kbit

        Total delay is 20100 microseconds

        Reliability is 255/255

        Load is 1/255

        Minimum MTU is 1500

        Hop count is 1



DC_L3_SW#sh ip route 10.177.44.186

Routing entry for 10.177.44.186/32

  Known via "eigrp 999", distance 90, metric 2172416, type internal

  Redistributing via eigrp 999

  Last update from 10.177.43.70 on Port-channel2, 00:33:30 ago

  Routing Descriptor Blocks:

  * 10.177.43.70, from 10.177.43.70, 00:33:30 ago, via Port-channel2

      Route metric is 2172416, traffic share count is 1

      Total delay is 20100 microseconds, minimum bandwidth is 1544 Kbit

      Reliability 255/255, minimum MTU 1500 bytes

      Loading 1/255, Hops 1


DC_L3_SW#sh ip eigrp topology 10.177.44.17 6/30

EIGRP-IPv4:(999) (AS 999): Topology Default-IP-Routing-Table(0) entry for 10.177.44.176/30

  State is Passive, Query origin flag is 1, 1 Successor(s), FD is 1764352

  Descriptor Blocks:

  10.177.43.66 (Port-channel1), from 10.177.43.66, Send flag is 0x0

      Composite metric is (1764352/1761792), Route is Internal

      Vector metric:

        Minimum bandwidth is 2048 Kbit

        Total delay is 20100 microseconds

        Reliability is 255/255

        Load is 3/255

        Minimum MTU is 1500

        Hop count is 1


DC_L3_SW1#sh ip     route 10.177.44.178

Routing entry for 10.177.44.176/30

  Known via "eigrp 999", distance 90, metric 1764352, type internal

  Redistributing via eigrp 999

  Last update from 10.177.43.66 on Port-channel1, 00:32:34 ago

  Routing Descriptor Blocks:

  * 10.177.43.66, from 10.177.43.66, 00:32:34 ago, via Port-channel1

      Route metric is 1764352, traffic share count is 1

      Total delay is 20100 microseconds, minimum bandwidth is 2048 Kbit

      Reliability 255/255, minimum MTU 1500 bytes

      Loading 3/255, Hops 1


*******************************************************************************************************************************************************


I have tried clearing ip eigrp neighbors and removing the interfaces and connecting back to the L3 core switch. But still the bandwidth of the neighbor 10.177.44.186 still shows as 1544 kbps, where i have already changed the Bandwidth on the Serial Links to 2048kbps.


Please can help me know how to solve this problem!


Regards,

Keshava Raju

Laurent Aubert Thu, 12/03/2009 - 06:22
User Badges:
  • Cisco Employee,

Hi,


Did you check the Bw and delay of all port-channel interfaces as well ?


I agree it's not normal. Did you try also a clear ip route * ? Are both core switched running the same version ?


Another option, could be to use bgp next-hop-self feature on your eBGP routers toward your core switch.


Thanks


Laurent.

Actions

This Discussion