Re: BGP Multi-homed routing help

rahman.faisal · ‎06-23-2006

Hello,

I've a Cisco 3600 series router connected to two ISP via BGP. One being over satellite and the other through fiber. This is how the BGP config looks:

router bgp 100

bgp log-neighbor-changes

network 192.168.48.0

network 192.168.48.0 mask 255.255.240.0

aggregate-address 192.168.48.0 255.255.240.0 summary-only

neighbor 172.16.39.3 remote-as 200

neighbor 172.16.39.3 description Satellite BGP connection 2

neighbor 172.16.39.3 ebgp-multihop 10

neighbor 172.16.39.3 weight 100

neighbor 172.16.39.4 remote-as 200

neighbor 172.16.39.4 description Satellite BGP connection

neighbor 172.16.39.4 ebgp-multihop 10

neighbor 172.16.39.4 weight 100

neighbor 10.111.207.173 remote-as 300

neighbor 10.111.207.173 description Fiber BGP connection

neighbor 10.111.207.173 weight 200

Now the problem is even with the weight set to 200, downstream data is mostly comming (about 90%) comming through the satellite downlink. But I want to have

1. both the upload and download come through the fiber link first,

2. if fiber congested then load share with satellite

3. if fiber fails, switch to satellite.

Please suggest how to achieve the above.

Thanks in advance.

kamlesh.sharma · ‎06-23-2006

1. both the upload and download come through the fiber link first,

A1-you need to set high local preference updates coming in and lower med updates going out from fiber link.

A2- You can use route-map and set AS prepend to make a path connected to satelite as backup say for example

neighbor 172.16.39.3 remote-as 200 route-map foo out

!--- It appends AS 100 to the BGP updates sent to AS 400

!--- in order to make it a backup for the ISP-A to R1 path.

!

access-list 1 permit x.x.x.x

route-map foo permit 10

match ip address 1

set as-path prepend 100 100

2. if fiber congested then load share with satellite

A1- possible by running HSRP but in ur case you have only one router.

3. if fiber fails, switch to satellite.

by default it will happen if you have peership from both the link.

olorunloba · ‎06-24-2006

The weight configuration is to modify your inbound routing update, this will only affect your uplink traffic. To affect your downlink traffic, you need to modify your outbound routing update. With BGP, there 2 ways to do this, either using MED or using AS path prepend.

MED works only if both paths are to the same AS. AS path prepend is often the preferred option. A common question is how many number of AS number should be appended. You will need to monitor this, and change the number till you see a desired utilisation levels on your link.

Check the following link

http://www.cisco.com/en/US/tech/tk365/technologies_configuration_example09186a00800945bf.shtml

2. To achieve this, consider running OER, Optimised Edge Routing. Check the following link.

http://www.cisco.com/en/US/netsol/ns471/networking_solutions_package.html

3. The former BGP configs will hsve achieved this.

rahman.faisal · ‎06-24-2006

Thank you Kamlesh and Olorunloba. I have used AS path prepend and achived about 33% of the traffic to come through the fiber link and the rest is still coming through the satellite. Also I have set local preference of 1000 to the fiber and 50 to the satellite inbound. Syncronization is on.

This is the new configuration:

router bgp 100

bgp log-neighbor-changes

network 192.168.48.0

network 192.168.48.0 mask 255.255.240.0

aggregate-address 192.168.48.0 255.255.240.0 summary-only

neighbor 172.16.39.3 remote-as 200

neighbor 172.16.39.3 description satellite BGP connection 2

neighbor 172.16.39.3 ebgp-multihop 10

neighbor 172.16.39.3 weight 100

neighbor 172.16.39.3 route-map satellite-in in

neighbor 172.16.39.3 route-map foo out

neighbor 172.16.39.4 remote-as 200

neighbor 172.16.39.4 description satellite BGP connection

neighbor 172.16.39.4 ebgp-multihop 10

neighbor 172.16.39.4 weight 100

no neighbor 172.16.39.4 activate

neighbor 10.111.207.173 remote-as 300

neighbor 10.111.207.173 description fiber BGP connection

neighbor 10.111.207.173 weight 200

neighbor 10.111.207.173 route-map fiber-in in

maximum-paths 4

!

route-map satellite-in permit 10

set local-preference 50

!

route-map fiber-in permit 10

set local-preference 1000

!

route-map foo permit 10

set as-path prepend 100 100 100 100 100 100 100 100 100 100

This the BGP trace from APNIC site:

From Brisbane:

Brisbane Local time [Sun Jun 25 09:34:22 2006]

BGP routing table entry for 202.92.48.0/20, version 23915362

Paths: (2 available, best #2, table Default-IP-Routing-Table)

Not advertised to any peer

1221 4637 3356 200 100 100 100 100 100 100 100 100 100 100 100, (aggregated by 100 10.111.207.174)

203.14.8.16 (metric 2) from 203.14.8.16 (203.14.8.16)

Origin IGP, localpref 80, valid, external, atomic-aggregate

Dampinfo: penalty 192, flapped 3 times in 00:44:51

1221 4637 3356 200 100 100 100 100 100 100 100 100 100 100 100, (aggregated by 100 10.111.207.174)

202.12.29.79 (metric 2) from 202.12.29.79 (202.12.29.79)

Origin IGP, metric 0, localpref 500, valid, internal, atomic-aggregate, best

From Tokyo:

Brisbane Local time [Sun Jun 25 09:36:15 2006]

BGP routing table entry for 202.92.48.0/20, version 2540172

Paths: (2 available, best #2, table Default-IP-Routing-Table)

Advertised to non peer-group peers:

193.0.4.28 202.12.29.64 202.249.2.143 202.249.2.166

2497 6453 200 100 100 100 100 100 100 100 100 100 100 100, (aggregated by 100 10.111.207.174)

202.249.2.169 from 202.249.2.169 (202.232.1.98)

Origin IGP, localpref 100, valid, external, atomic-aggregate

2516 1239 702 702 300 100, (aggregated by 100 10.111.207.174)

202.249.2.110 from 202.249.2.110 (203.181.99.231)

Origin IGP, localpref 100, valid, external, atomic-aggregate, best

By the way I've IOS Version 12.1(5)T7, so no OER support.

Please suggest what should I do.

Thanks.

kamlesh.sharma · ‎06-24-2006

can you put your show ip bgp output so that we will be able to help you well.

i think you are done with your config.

As in your output given above shows from brisbon it is reachable via AS 200 but from tokyo it reachable via As 300.

this time your all the traffic should go to fiber link and coming from both the link just check.

please rate if helps

Kamlesh

rahman.faisal · ‎06-25-2006

Below are the results of ip bgp and neighbors

#sh ip bgp

BGP table version is 28, local router ID is 10.111.207.174

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*> 0.0.0.0 10.111.207.173 1000 200 300 702 i

* 172.16.39.3 50 100 200 i

s> 192.168.48.0 0.0.0.0 0 32768 i

*> 192.168.48.0/20 0.0.0.0 32768 i

#sh ip bgp neighbors 172.16.39.3

core-danl#sh ip bgp neighbors 172.16.39.3

BGP neighbor is 172.16.39.3, remote AS 200, external link

Description: satellite BGP connection 2

BGP version 4, remote router ID 172.16.39.3

BGP state = Established, up for 05:55:13

Last read 00:00:14, hold time is 180, keepalive interval is 60 seconds

Neighbor capabilities:

Route refresh: advertised and received(new)

Address family IPv4 Unicast: advertised and received

Received 1372 messages, 0 notifications, 0 in queue

Sent 1427 messages, 1 notifications, 0 in queue

Route refresh request: received 0, sent 8

Default minimum time between advertisement runs is 30 seconds

Default weight 100

For address family: IPv4 Unicast

BGP table version 28, neighbor version 28

Index 1, Offset 0, Mask 0x2

Inbound path policy configured

Outbound path policy configured

Route map for incoming advertisements is satellite-in

Route map for outgoing advertisements is foo

1 accepted prefixes consume 36 bytes

Prefix advertised 5, suppressed 0, withdrawn 1

Number of NLRIs in the update sent: max 1, min 0

Connections established 5; dropped 4

Last reset 11:53:04, due to Neighbor deleted

External BGP neighbor may be up to 10 hops away.

Connection state is ESTAB, I/O status: 1, unread input bytes: 0

Local host: 192.168.48.129, Local port: 179

Foreign host: 172.16.39.3, Foreign port: 48164

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x1A2D70A4):

Timer Starts Wakeups Next

Retrans 383 20 0x0

TimeWait 0 0 0x0

AckHold 361 177 0x0

SendWnd 0 0 0x0

KeepAlive 0 0 0x0

GiveUp 0 0 0x0

PmtuAger 0 0 0x0

DeadWait 0 0 0x0

iss: 93155242 snduna: 93162430 sndnxt: 93162430 sndwnd: 16384

irs: 3203800665 rcvnxt: 3203807597 rcvwnd: 16061 delrcvwnd: 323

SRTT: 709 ms, RTTO: 957 ms, RTV: 248 ms, KRTT: 0 ms

minRTT: 524 ms, maxRTT: 1068 ms, ACK hold: 200 ms

Flags: passive open, nagle, gen tcbs

Datagrams (max data segment is 536 bytes):

Rcvd: 731 (out of order: 0), with data: 361, total data bytes: 6931

Sent: 553 (retransmit: 20), with data: 362, total data bytes: 7187

rahman.faisal · ‎06-25-2006

#sh ip bgp neighbors 10.111.207.173

BGP neighbor is 10.111.207.173, remote AS 300, external link

Description: fiber BGP connection

BGP version 4, remote router ID 10.111.195.214

BGP state = Established, up for 00:15:50

Last read 00:00:09, hold time is 30, keepalive interval is 10 seconds

Neighbor capabilities:

Route refresh: advertised and received(new)

Address family IPv4 Unicast: advertised and received

Received 43895 messages, 5 notifications, 0 in queue

Sent 43947 messages, 14 notifications, 0 in queue

Route refresh request: received 0, sent 10

Default minimum time between advertisement runs is 30 seconds

Default weight 200

For address family: IPv4 Unicast

BGP table version 28, neighbor version 28

Index 2, Offset 0, Mask 0x4

Inbound path policy configured

Route map for incoming advertisements is fiber-in

1 accepted prefixes consume 36 bytes

Prefix advertised 58, suppressed 0, withdrawn 28

Number of NLRIs in the update sent: max 1, min 0

Connections established 24; dropped 23

Last reset 00:16:35, due to BGP Notification sent, hold time expired

Connection state is ESTAB, I/O status: 1, unread input bytes: 0

Local host: 10.111.207.174, Local port: 11045

Foreign host: 10.111.207.173, Foreign port: 179

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x1A2F1DDC):

Timer Starts Wakeups Next

Retrans 121 20 0x0

TimeWait 0 0 0x0

AckHold 100 42 0x0

SendWnd 0 0 0x0

KeepAlive 0 0 0x0

GiveUp 0 0 0x0

PmtuAger 0 0 0x0

DeadWait 0 0 0x0

iss: 2861998644 snduna: 2862000658 sndnxt: 2862000677 sndwnd: 15833

irs: 1678389569 rcvnxt: 1678391540 rcvwnd: 15909 delrcvwnd: 494

SRTT: 509 ms, RTTO: 1816 ms, RTV: 1307 ms, KRTT: 0 ms

minRTT: 0 ms, maxRTT: 1496 ms, ACK hold: 200 ms

Flags: higher precedence, nagle

Datagrams (max data segment is 1460 bytes):

Rcvd: 206 (out of order: 0), with data: 101, total data bytes: 1970

Sent: 167 (retransmit: 20), with data: 101, total data bytes: 2032

Thanks.

Edison Ortiz · ‎06-25-2006

[quote]

neighbor 10.111.207.173 remote-as 300

neighbor 10.111.207.173 description fiber BGP connection

neighbor 10.111.207.173 weight 200

neighbor 10.111.207.173 route-map fiber-in in

[/quote]

You are missing 'ebgp-multihop' for this neighbor.

olorunloba · ‎06-26-2006

Is this BGP negihbour 172.16.39.4 up?.

You have two neighbours in As 200 i.e. 172.16.39.3 and 172.16.39.4. You however are prepending to 172.16.39.3 and not to 172.16.39.4. Depending on the setup of As 200, your prepending might not have the full desired result. Apply the foo route-map also to 172.16.39.4, and check again what your utilisation is.

kamlesh.sharma · ‎06-26-2006

Your all the traffic is going outside via fiber link only as default route is best and installed in routing table but i wondered why it is showing only 702 i AS no i suppose you have connectivity to MCI international. but if you have another ip block from VSNL or from other service provider your traffic for that ip block will always come from there link until and unless your link goes down conected to that provider. next i would like to know what traffic is coming in and what traffic is going out so that we can have broad view of traffic flow so that we will be able to differentiate what traffic should come froom where and where we need to have load balancing. waiting for ur input.

please rate if it helps

HTH

Kamlesh