Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

Help with BGP

Hi All,

I have a router with a single physical interface connecting to the outside world and talking BGP to two ISPs.

Here's the BGP config:

router bgp 65100
no synchronization
bgp log-neighbor-changes
network NETWORK mask 255.255.255.128
neighbor x.x.x.x remote-as 11830 (link to secondary connection)
neighbor x.x.x.x route-map METRIC in
neighbor y.y.y.y remote-as 11830 (link to primary connection)
neighbor y.y.y.y route-map METRIC in
maximum-paths 2
no auto-summary

According to the IP routing table, the router has two default gateways (one for each BGP), doing load balancing:

B*   0.0.0.0/0 [20/0] via y.y.y.29, 1w1d
                        [20/0] via x.x.x.5, 14:03:48

According to the BGP table, the preferred path should be y.y.y.129

   Network          Next Hop            Metric LocPrf Weight Path
*  0.0.0.0          x.x.x.5                  0             0 11830 i
*>                     y.y.y.29                 0             0 11830 i

The problem that I'm having is that if we disconnect the second link (connection to x.x.x.5), everything stops working.
If I do a traceroute from the local network, we get out to the Internet using x.x.x.5, but according to the BGP table
the preferred path should be through y.y.y.29 (which is supposed to be the primary link).

My goal is to have all traffic going out via y.y.y.29 and if it fails to go over x.x.x.5 (and then come back to the primary
connnection when the primary link comes back).

How can I check what's going on?

Federico.

3 ACCEPTED SOLUTIONS

Accepted Solutions
Hall of Fame Super Blue

Re: Help with BGP

Federico

It seems that the router first try to reach the x.x.x.5, then y.y.y.29, but then again x.x.x.5
Is this a problem with the BGP peer y.y.y.29?

What does the maximum-paths 2 under BGP accomplishes? Load balancing?

Shouldn't it prefer the route mark as ''best'' under BGP?


By default even if there are multiple equal cost paths to the same destination BGP will only use one of them. By adding the maximum-paths 2 command you have told BGP it can use 2 equal cost paths. Hence the reason it tries x.x.x.5 then y.y.y.29 then x.x.x.5 etc..

If you want it to always use just one of the default-routes then you should look to remove the maximum-paths command but only if you are sure it is not being used for some other destinations other than the default-route.

It will then only use one of the paths. As they are both equal cost though it could choose either so you will need to make sure it picks the y.y.y.29. You can modify the weight for this if you want.

Jon

Hall of Fame Super Blue

Re: Help with BGP

coto.fusionet wrote:

Yes.


We had a problem with a fiber and we physically disconnect the link to one BGP peer.
When we did this, all Internet traffic stopped working.
As soon as we connected the link back again (both BGP links up), then everything went back to normal.


We were surprised since, when disconnecting one BGP connection, the other one should assume the traffic.

I am going to do the test again as soon as possible to check the traceroute (when the link is down) and let you know.
The BGP configuration seems fine.

Federico.

Federico

It might also be worth checking with the ISP to see what they see when you disconnect one of the links. I'm assuming that they dynamically receive your networks via EBGP advertisements because you do have a network statement in your BGP config but if they were using statics at their end for some of your networks then their routers might not realise your end has gone down so outbound traffic from you goes the right way but return traffic does not.

Also how long are you waiting after disconnection. Bear in mind BGP is not the quickest to failover and you have not modified your BGP timers so you could be looking at minutes before the ISP end updated and removed the route to your networks back via the disconnected link.

Jon

New Member

Re: Help with BGP

Hi!,

Along with Timers you can use weight on the primary link so that your internal traffic will prefer this primary link.

If the primary link goes down traffic will automatically be diverted through the second link.

If the Primary link comes back up again traffic will automatically choose the preimary link with the secondary as back-up

Regards,

Ranjit

14 REPLIES
Hall of Fame Super Blue

Re: Help with BGP

Federico

You have 2 ISPs, are they both using the same AS number ??? ie. 11830

We need to see the route-map configuration. Also can you post the output of "sh ip route | include 0.0.0.0"

Also, if you want to prefer one route over the other why have you configured "maximum-paths 2" under the BGP config ?

Jon

Re: Help with BGP

I'm sorry, is the same ISP with two BGP peers.

The route-map configuration is just this:

route-map METRIC permit 10
match ip address 101
access-list 101 permit ip 201.193.214.0 0.0.0.127 any

Output of sh ip route | include 0.0.0.0:

r-internet-ice#sh ip route | include 0.0.0.0
Gateway of last resort is y.y.y.29 to network 0.0.0.0
B*   0.0.0.0/0 [20/0] via y.y.y.29, 1w

Also,
When I do a trace from the Internet router, this is what I get:

r-internet-ice#trace 4.2.2.2

Type escape sequence to abort.
Tracing the route to 4.2.2.2

  1 x.x.x.5 [AS 11830] 8 msec
    y.y.y.29 [AS 11830] 0 msec
    x.x.x.5 [AS 11830] 4 msec
  2 x.x.x.97 [AS 11830] 0 msec 0 msec 4 msec
  3 144.223.245.133 [AS 11830] 40 msec 40 msec 40 msec
  4 144.232.2.241 [AS 11830] 40 msec 44 msec 40 msec
  5 4.68.62.77 [AS 11830] 56 msec 56 msec 56 msec
  6 4.69.138.126 [AS 11830] 64 msec 56 msec 68 msec
  7 4.69.140.142 [AS 11830] 56 msec 68 msec 56 msec
  8 4.68.103.34 [AS 11830] 56 msec
    4.68.103.2 [AS 11830] 56 msec
    4.68.103.34 [AS 11830] 56 msec
  9 4.2.2.2 [AS 11830] 60 msec 56 msec 56 msec

It seems that the router first try to reach the x.x.x.5, then y.y.y.29, but then again x.x.x.5
Is this a problem with the BGP peer y.y.y.29?

What does the maximum-paths 2 under BGP accomplishes? Load balancing?

Shouldn't it prefer the route mark as ''best'' under BGP?

Federico.

Re: Help with BGP

I've mentioned it was a single physical interface, sorry its two interfaces on the router.

So, the scenario is like this:

Single router having two BGP peer with the same ISP (using two outside interfaces)

Federico.

Re: Help with BGP

Also Jon, sorry for not posting the question correctly from start:

This is what we want:

We want to use both links to both BGP peers corresponding to the same ISP in load-balancing.

Since both links are 30M, we want to balance the traffic across both links.

But why if I disconnect the link to one BGP peer, everything stops working?

Sorry for the confusion, I hope its clear now.

Federico.

Hall of Fame Super Blue

Re: Help with BGP

Federico

How are you "disconnecting" the link to x.x.x.5 as you only have one physical link ?

Also what is the physical link, is it ethernet ?

Jon

Hall of Fame Super Blue

Re: Help with BGP

Federico

It seems that the router first try to reach the x.x.x.5, then y.y.y.29, but then again x.x.x.5
Is this a problem with the BGP peer y.y.y.29?

What does the maximum-paths 2 under BGP accomplishes? Load balancing?

Shouldn't it prefer the route mark as ''best'' under BGP?


By default even if there are multiple equal cost paths to the same destination BGP will only use one of them. By adding the maximum-paths 2 command you have told BGP it can use 2 equal cost paths. Hence the reason it tries x.x.x.5 then y.y.y.29 then x.x.x.5 etc..

If you want it to always use just one of the default-routes then you should look to remove the maximum-paths command but only if you are sure it is not being used for some other destinations other than the default-route.

It will then only use one of the paths. As they are both equal cost though it could choose either so you will need to make sure it picks the y.y.y.29. You can modify the weight for this if you want.

Jon

Re: Help with BGP

The router has two physical connections to the ISP (I corrected earlier).
Each interface handles a BGP peer.
Both connections are Ethernet.

As a summary Jon,
We need to load-balance traffic across both links (seems to be happening now)...
But we need the either link to assume all traffic if one disconnects (this is not happening)...

Thank you for your help.

Hall of Fame Super Blue

Re: Help with BGP

Right, that makes more sense

So when you disconnect x.x.x.5 are you physically shutting the interface down ?

And once you have done this what does a traceroute show ie. do you see it going out via y.y.y.29 ?

Your traceroute from before is normal if you have 2 equal cost paths in the routing table ie. traceroute will use them both.

Also you are advertising just one network to both ISPs ?

Jon

Re: Help with BGP

Yes.


We had a problem with a fiber and we physically disconnect the link to one BGP peer.
When we did this, all Internet traffic stopped working.
As soon as we connected the link back again (both BGP links up), then everything went back to normal.


We were surprised since, when disconnecting one BGP connection, the other one should assume the traffic.

I am going to do the test again as soon as possible to check the traceroute (when the link is down) and let you know.
The BGP configuration seems fine.

Federico.

Hall of Fame Super Blue

Re: Help with BGP

coto.fusionet wrote:

Yes.


We had a problem with a fiber and we physically disconnect the link to one BGP peer.
When we did this, all Internet traffic stopped working.
As soon as we connected the link back again (both BGP links up), then everything went back to normal.


We were surprised since, when disconnecting one BGP connection, the other one should assume the traffic.

I am going to do the test again as soon as possible to check the traceroute (when the link is down) and let you know.
The BGP configuration seems fine.

Federico.

Federico

It might also be worth checking with the ISP to see what they see when you disconnect one of the links. I'm assuming that they dynamically receive your networks via EBGP advertisements because you do have a network statement in your BGP config but if they were using statics at their end for some of your networks then their routers might not realise your end has gone down so outbound traffic from you goes the right way but return traffic does not.

Also how long are you waiting after disconnection. Bear in mind BGP is not the quickest to failover and you have not modified your BGP timers so you could be looking at minutes before the ISP end updated and removed the route to your networks back via the disconnected link.

Jon

Re: Help with BGP

Jon,

We are advertising just one public range in BGP to the ISP.

What are good BGP timers?

Federico.

Hall of Fame Super Blue

Re: Help with BGP

coto.fusionet wrote:

Jon,

We are advertising just one public range in BGP to the ISP.

What are good BGP timers?

Federico.

Federico


I have used timers of 10 & 30 rather than the default of 60 & 180 and they worked fine.

Jon

New Member

Re: Help with BGP

Hi!,

Along with Timers you can use weight on the primary link so that your internal traffic will prefer this primary link.

If the primary link goes down traffic will automatically be diverted through the second link.

If the Primary link comes back up again traffic will automatically choose the preimary link with the secondary as back-up

Regards,

Ranjit

Re: Help with BGP

Thank you my friends.

Federico.

372
Views
0
Helpful
14
Replies