iBGP behaviour

Unanswered Question
May 20th, 2008

Hi all,

I have a question regarding our BGP configuration that's getting me mad. We have two 7604 doing eBGP with three providers and iBGP between them.

This is our config for the iBGP peeering (some data changed to protect the innocent)

router1:

router bgp xxxx

no synchronization

no bgp fast-external-fallover

bgp log-neighbor-changes

network x.x.x.x mask 255.255.192.0

neighbor 10.1.1.253 remote-as xxxx

neighbor 10.1.1.253 description iBGP_R01_R02

neighbor 10.1.1.253 update-source Loopback0

neighbor 10.1.1.253 version 4

neighbor 10.1.1.253 next-hop-self

neighbor 10.1.1.253 soft-reconfiguration inbound

no auto-summary

router2:

router bgp xxxx

no synchronization

no bgp fast-external-fallover

bgp log-neighbor-changes

network x.x.x.x mask 255.255.192.0

neighbor 10.1.1.249 remote-as xxxx

neighbor 10.1.1.249 update-source Loopback0

neighbor 10.1.1.249 version 4

neighbor 10.1.1.249 next-hop-self

neighbor 10.1.1.249 soft-reconfiguration inbound

no auto-summary

Well, when I issue a sh ip bgp summ I've got that any of the iBGP peers have advertised only a fraction of the total BGP table routes, i.e., 180000 one and 30000 the other. In fact, if I do a sh ip bgp x.x.x.x, I can see that a prefix learned by both routers through the three carriers is passed only one way: from route1 to router2 OR from router2 to router1, but not both ways. In fact, I can see that the route marked as "best" in one router BGP table is passed to the other router, but the same route is not marked as best, is ignored and not passed to the iBGP peer.

Result of this is that I have not a good route decision, because both routes are not exchanging all the information they need to do a proper election.

¿Any help? ¿Some misconfig? ¿Am I loosing something in the way iBGP operates?

thank you

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 3.8 (7 ratings)
Loading.
royalblues Tue, 05/20/2008 - 03:52

BGP always advertises only the best path to its neighbors and not all the paths. so the protocol is acting as it should

you will have to tweak the different BGP attributes so that the router does the proper election

Narayan

laloperez Tue, 05/20/2008 - 04:07

But, if I've got a route to 1.1.1.1 through router1 and another route to the same 1.1.1.1 through router2, how can router2 know if its route is best or worse than that of router1? Or is it that the info in the BGP table is just passed through the BGP decission process, so the worst routes are not present?

thank you

royalblues Tue, 05/20/2008 - 04:26

If Router 2 in the above case receives 2 routes to the same destination, then it will run the BGP algorithm and chose the best path.

Only this path is advertised to the neighbors and not the other one

for eg say r1 --r2--r3 and both r1 and r3 originate 1.1.1.0/24 and advertise it to R2. R2 chooses the path via R1 as per the BGP decision process. Now R2 will advertise only the best path to R3 but R3 prefers its locally originated route.

Since the route to 1.1.1.0/24 received from R3 is not the best, it will not be sent to R1 and can be seen by sh ip bgp nei advertised-routes

This is also the reason why you would be seeing 18000 routes in one vs 30000 routes in other

HTH

Narayan

laloperez Tue, 05/20/2008 - 06:37

Sorry, I didn't explained well. I have just two routers, one connected to two providers and the other to just one. I.e.:

ISP1 ISP2 ISP3

\ / |

R1-----R2

So, R1 see, f.eg., 1.1.1.0/24 through ISP1 and ISP2, and R2 sees the same 1.1.1.0/24 through ISP3. R1 decides that ISP1 is the best route and marks it accordingly. That route is passed to R2. Ok. But, I presume that, before the exchange, R2 sees the best route to 1.1.1.0/24 through its only provider, ISP3, and must mark it as best and pass it to R1. After the exchange, maybe ISP1 remains as the best route, maybe not. But for that to be done, R1 must pass its best route and R2 its one.

But, if I do a sh ip bgp nei I don't see that supposed simmetry.

lamav Tue, 05/20/2008 - 06:26

Lalito :-)

Check it out, papi....

You have to understand a few basic concepts, right?

First, if a BGP router learns of a certain prefix from multiple BGP speakers, with whom it has a peering relationship, it will pass all the prefixes through the BGP Best Path Selection Algorithm and select the BEST path.

Second, all learned prefixes will be in the BGP table (sh ip bgp...) and all of them are candidates to be placed in the IP routing table, but ONLY the BEST path will actually be placed in the IP routing table and advertised out.

Now, if I have the scenario you describe above, R1 and R2 will receive prefix advertisements from ALL BGP speakers, including from each other, and each router will select the BEST path only and place that path in the IP routing table. The BGP table, however, will have ALL the BGP-learned prefixes.

Lastly, something to think about...iBGP has an administrative distance of 200, while eBGP has an administratve distance of only 20. When R1 and R2 advertise their routes to each other, they are going to reject each other's advertisements IF they have comparable eBGP-learned prefixes in their BGP table.

So, if R1 learns of 1.1.1.0/24 from Sprint, AT&T and Broadwing, and Sprint has the BEST path, it will place that route through Sprint in its IP routing table and advertise it to R2 through iBGP. BUT, check it out, R2 ALSO has a route to 1.1.1.0 in its routing table that it learned from Sprint, and it will advertise it to R1. R1 will reject that route without even putting it through the BGP best path selection process because that route will have an AD of 200 and the other routes have an AD of 20. By the same token, R2 will reject R1's comparable advertisements for the same reason.

[EDIT] Remember, a router will place the route with the lower administatve distance in its routing table. [EDIT]

I hope this has helped you...

Victor

laloperez Tue, 05/20/2008 - 07:03

Hmmm, do you know me? :O

Anyway, I knew all you are saying about the installation of the route in the routing table, and the process involved in it. But in your example both routers have a route through Sprint. In my case R2 knows that route from ISP3 and R1 learns it from ISP1 and ISP2. So, I think they are not exactly comparable at first. See my previous comment to a Narayan.

lamav Tue, 05/20/2008 - 07:09

Lalo:

It doesnt matter. You should be paying attention to the general principles involved. Nothing changes with the scenario you gave. R1 and R2 will select the best path from the eBGP advertisements, advertise them to each other through iBGP, and comparable eBGP routes will be ignored.

What dont you understand?

Victor

laloperez Tue, 05/20/2008 - 08:10

>>What dont you understand?

Just why the routes from the other router didn't appeared in one BGP table as they did in the other. I always thought (till now) that the best route information was bi-directional and remained in the BGP table after the selection.

Just a question. If R1 has a route to 1.1.1.0 with as-path 30 10, for example, and R2 has one like 50 10, supposed all other attributes equal, BGP table will show BOTH routes in BOTH routers, each one with its own "best" installed in the routing table and with packets going out one way or the other based in wich router entered the packet. Am I rigth?

So, only when there is a winner among all the routers for a particular prefix, the best route appears in the BGP table of all routers, and the routes from the other routers don't. Right?

lamav Tue, 05/20/2008 - 08:56

Lalo, maybe Im not the one to help you because I cant seem to pinpoint your dilemma.

Let me give it a last ditch effort and reiterate some points from a different angle.

When we talk about a router selecting the best path, we are talking about the best path relative to that router. R1's best path to a prefix may not be R2's best path.

In your case, R1 and R2 are going to have all the prefixes learned through BGP in their BGP tables, with only the best installed in their IP routing table. Remember that the ISP routers that are advertising routes to R1 and R2 are only advertising THEIR best paths, not all the paths in their BGP tables. R1 and R2 will never see those.

I have a problem with this question:

"So, only when there is a winner among all the routers for a particular prefix, the best route appears in the BGP table of all routers, and the routes from the other routers don't. Right?"

Its confusing. You make it sound like all the BGP speakers are going to agree on one prefix path as the best for a particular network, and then that route is going to appear in everyones routing table. It does not work that way.

HTH

If not, I would recommend that you wait for someone else to give you a hand and fill in the blanks Im failing to fill.

Thanks

Victor

lamav Tue, 05/20/2008 - 09:19

Lalo:

I will correct one thing that I said, although I dont think it contributed to your confusion.

"So, if R1 learns of 1.1.1.0/24 from Sprint, AT&T and Broadwing, and Sprint has the BEST path, it will place that route through Sprint in its IP routing table and advertise it to R2 through iBGP. BUT, check it out, R2 ALSO has a route to 1.1.1.0 in its routing table that it learned from Sprint, and it will advertise it to R1. R1 will reject that route without even putting it through the BGP best path selection process because that route will have an AD of 200 and the other routes have an AD of 20. By the same token, R2 will reject R1's comparable advertisements for the same reason."

The part in bold is incorrect, I misspoke. BGP will not differentiate between iBGP and eBGP unless all higher order precedence tests for path selection do not render a best path.

So, if R1 advertises a route to R2 with, say, a higher local preference than all other routes learned for the same prefix by R2 through BGP, R2 WILL accept that route as being the best path. It will not automatically reject it because of iBGP's higher AD.

If, however, the comparison was made between, say, OSPF or EIGRP, and iBGP, then AD will play a role in the selection process.

Of course, none of this changes the general principles I have described to you and which you say you already understand anyway. But I did want to clear that one piece of misinformation I gave you. I apologize if that confused you more.

Victor

lamav Tue, 05/20/2008 - 09:39

Lalo:

R2 determined that R1's path advertisement for 212.172.0.0 was best because the other sources are advertising a higher metric. The higher the metric, the LESS desirable the route.

from router 2:

sh ip bgp 212.172.0.0

BGP routing table entry for 212.172.0.0/16, version 12108320

Paths: (3 available, best #1, table Default-IP-Routing-Table)

Not advertised to any peer

42612 6453 3257 12312, (received & used)

10.1.1.249 (metric 2) from 10.1.1.249 (10.1.1.249)

Origin IGP, metric 0, localpref 100, valid, internal, best

42612 174 3257 12312

149.6.150.69 from 149.6.150.69 (38.28.4.32)

Origin IGP, metric 21001, localpref 100, valid, external

Community: 11424364 11425276

174 3257 12312, (received-only)

149.6.150.69 from 149.6.150.69 (38.28.4.32)

Origin IGP, metric 21001, localpref 100, valid, external

Community: 11424364 11425276

Does this help you now?

Victor

laloperez Wed, 05/21/2008 - 01:41

Yes, I can see why the route through 6453 is chosen instead of the one through 174. But (sorry :D) what I didn't understand was that the BGP table in R1 reflects already the fact that a best route is elected in itself, passed to R2, and as R2, after receiving update from R1, sees no better option to offer to R1, declines to send any of its own routes. Maybe in the initial interchange I could see for a brief moment the 174 route in R1, until the election in R1 is made?

An yes, your assertion about the admin distance puzzled me a bit :)

Anyway, I think that the issue is resolved and we are just running circles around the same issue :D. But I've learnt a bit more of the inners of BGP.

Thank you very much.

milan.kulik Wed, 05/21/2008 - 04:58

Hi Lalo,

what about this explanation:

In the second case, R2 received 212.172.0.0/16 prefix from R1 first, installed to the BGP table as the best one and THEN received the same prefix from his ISP. As the ISP prefix was worse, was not sent to to R1 (only the best would be).

In the first case, R2 received the 62.36.0.0/19 prefix from the ISP first, installed in to his BGP table as best and sent to R1.

After receiving a better path to 62.36.0.0/19 advertized from R1, installed to the BGP table but did not send back, of course.

R1 received the prefixes from his ISP first in both cases.

Does this make a sense?

BR,

Milan

laloperez Wed, 05/21/2008 - 06:14

Absolutely, yes :). In fact, this was my conclusion in the previous post. I think that in no place in traditional BGP docs is mentioned anything about this timing issue. From this, my initial confusion :)

Thank you all, guys. This conversation made me think deeper into the innards of BGP, and I think I understand better how it really works.

lamav Wed, 05/21/2008 - 07:28

Lalo, after reading your post this morning, i see a bit better what your issue involves.

I think you want to see BGP converging so that you can understand the process. Thats great because thats how you really get a good understanding of how the protocol works.

When you can -- during off hours or in a lab -- use some debug commands to see BGP in action as it converges and stabilizes.

Victor

laloperez Wed, 05/21/2008 - 07:52

Hi, Victor,

yes it's something like that. I can understand now that the BGP table i can see is a "stable" one, with all the interchanges made and such. In the first post I asumed that the lack of routes from any of the routers implied an incomplete decision making for the BGP process. But I didn't realize that for some routes I thought were incorrectly chosen based only on as-path and local-pref, there was a difference in the med sent from our peers and then the decision made sense. Now I have to manipulate that to improve that decision :)

I have a virtual lab (dynamips) and I'm going to give BGP a chance to self-explain :D

Best regards,

lamav Wed, 05/21/2008 - 08:22

Good luck, Lalo.

Sorry it took us so long to sync up, but we made it! :-)

Victor

And thanks for all the ratings.

Actions

This Discussion