BGP local pref

[Jon Marshall]

I understand what local pref does but i have always been slightly confused by one aspect of the topology it is used in.

If 2 IBGP peers are using local pref to influence which path to take is it best practice to have a direct physical link between the 2 routers or is it perfectly normal to have traffic being routed via the same interface it came in on eg.

1 L3 switch (SW1) shares a common subnet with WAN routers (R1 and R2). The WAN routers are using IBGP to peer.  Local pref has been setup so R1 is the preferred route for all networks learnt via EBGP.  Now i have seen a number of posts on CSC where the recommendation to influence the outgoing traffic is to use local pref. So SW1 can send traffic to either R1 and R2. If traffic goes to R2 that router will see the preferred path is via R1 and so it will have to reroute the traffic back out of the interface it was received on and send it to R1.

To my mind it would be far easier to redistribute the EBGP learned routes into OSPF (using type 1s from R1 and type 2s from R2) or EIGRP and influence the metrics. So SW1 always knows that R1 is the best path.

If there was physical dedicated connection between R1 and R2 that would make more sense to me.

I only ask because, as i say, i have seen a number of posts with a similiar setup where local pref was the recommendation i wondered whether i was missing something in terms of my understanding.

Any comments welcome.

Jon

[Richard Burts]

Jon

While there may be advantages in having a direct link between IBGP peers it is certainly not a requirement. And to my understanding there are not any particular implications of using local preference about whether the IBGP connection is direct or multi hop.

My reaction to your question is to think that we may have 2 goals in designing our networks

1) we want routing to be as efficient as possible

2) we want to implement policy about how traffic is forwarded

I believe that sometimes 2 requires some compromise in 1.

HTH

Rick

[Jon Marshall]

Rick

Thanks for the response.

I'm struggling to see why a common recommendation is to use IBGP in the example i gave. And i think your'e example outlines the issue i was trying to get to the bottom  of  ie.

Use IBGP in the above example and you meet the requirements of 2)  but not 1)

Redistibute BGP into an IGP while influencing the metrics and you get 1) and 2)

So why would you use IBGP in the scenario i outlined. I appreciate there are many other sceanrios within SP clouds etc. but i was really only concerned with this one and why i have seen a number of posts recommending local pref as a solution to the above when, to my mind, there are clearly better ways.

I thought i must be missing something.

Jon

[Richard Burts]

Jon

Let me then offer my opinion about the option to redistribute into the IGP vs using IBGP. If you are setting up a lab and your EBGP neighbor is advertising 20 prefixes to you then redistribution into the IGP is painless and makes some sense. But think about the Internet routing table. If you are running EBGP with an ISP who will advertise the entire Internet routing table, then what are the implications of redistributing that into your IGP? Do all of the routers inside your network have sufficient memory to maintain that many routes in their routing table? Do all of the routers inside your network have sufficient processing capability to maintain and search that big a routing table? Given the volitility of the Internet routing table do you want all of the routers inside your network to process all of the routing updates that are generated in the Internet?

I think it makes IBGP sound a bit better.

HTH

Rick

[Jon Marshall]

Rick

That is a very good point. I must admit i was thinking primarily of an MPLS WAN where you were receiving EBGP routes from your own remote sites and probably summarised at that so redistribution would not be such a big issue.

But it does raise the question if you were receiving full routes then why would you want all traffic to go via R1 in the above scenario. Surely you would want to use both routers to make a best path forwarding decision. If R2 was only to be used for backup then why bother giving it full routes ?

Following on from that, lets assume there are too many routes to redistribute. Lets also assume you are not receving a default route othrewise you could simply redistribute both of those and influence the metric.

So, in your opinion what would be a better solution -

1) use IBGP and accept traffic will be rerouted from R2 to R1

or

2) use static default routes on SW1 together with IP SLA so that traffic is always sent to the right router and no traffic needs to be rerouted.

Personally i have always favoured using dynamic routing protocols because they react to a failure anywhere down the line whereas IP SLA depends on just what IP you are tracking.

I appreciate you taking the time to answer these questions, sometimes it's just good to discuss these things so i can get it clear in my head.

Edit - i don't know why, i just feel uncomfortable with traffic having to be rerouted back the way it came from. Perhaps it's just me.

Jon

[John Blakley]

Jon,

We don't even use an igp in our network. I run ibgp between my 2 edge routers and our l3 switch. The two routers have a local pref set for certain routes. My primary is set as 150 for all routes except those that should go out R2 (DR traffic), and the same in reverse from R2. The switch sends it in the appropriate direction based on that.

My question that I've always wondered is why engineers use bgp on the edge, and then redistribute into an igp. Don't you lose the quick failover capability when bgp loses its peering? For example, we're on ethernet, so when my peering goes down on the provider side, my interface is still up and bgp has to time out. I have my hold times set fairly low, so it's not super horrible, but if I didn't change that it would be 3 minutes. Let's say for ease that my hold times were at 3 minutes, my timer counts down, and bgp is still trying to get the peering back from a flap. Let's also assume that I'm redistributing into ospf. OSPF still has routes, but wouldn't they be blackholed for 3 minutes while bgp is trying to recover? This is one reason I've never moved to an igp on the lan because I believe that bgp just plays better together.

HTH,
John

*** Please rate all useful posts ***

[Jon Marshall]

John

That is another very good point. In my last job we had the BGP timers set very low just because of the very issue you raise.

Couple of questions -

1) My primary is set as 150 for all routes except those that should go out R2 (DR traffic), and the same in reverse from R2. The switch sends it in the appropriate direction based on that.

But how does the switch know which router to send traffic to ?  Is this a L3 switch ? So DR traffic, how does the switch know which is a DR subnet and that R2 should be used for that ?

2) Without having to look through the docs when the BGP peering is lost the interface is still up/up in your setup. So are are you saying that BGP immediately flushes it's routes ? I don't quite follow ie. if BGP still thinks the link is up and is redistributing into OSPF then even without redistribution the router will still have it's BGP routes so it would not use the other router.

I think i may be misunderstanding something here ?

By the way, sorry you keep getting 3s in the dynamic routing posts, i keep trying to readjust as you have been more than helpful.

Edit - ignore 1 because i just reread and see you run IBGP on your L3 switch. I should have read more carefully.

Jon

[John Blakley]

Jon,

No problem at all

2) Without having to look through the docs when the BGP  peering is lost  the interface is still up/up in your setup. So are are  you saying that  BGP immediately flushes it's routes ? I don't quite  follow ie. if BGP  still thinks the link is up and is redistributing  into OSPF then even  without redistribution the router will still have  it's BGP routes so it  would not use the other router. 

BGP still has the routes in the routing table until the hold times expire, and then bgp would notice that the neighbor was down and remove the routes (unless it gets a notification before then). The problem is that, for example, if I was learning 192.168.1.0/24 from an ebgp peer and that peer went down, if I were redistributing the ebgp learned routes into ospf, ospf would see a route for 192.168.1.0/24 to the ospf neighbor that's also doing bgp/ospf redistribution. The neighbor has a down bgp peering, but it's not aware of it because the timers haven't expired, so that 192.168.1.0 route is still in the bgp and ospf table. I would think the pure ospf neighbor would still try to forward traffic until bgp timed out and would cause it to blackhole routes because bgp is down....man, that's confusing

HTH,
John

*** Please rate all useful posts ***

[Jon Marshall]

John

You're right it is confusing.

I guess what i was asking, and you may have answered but i am too dense to realise, is what makes IBGP any better in this instance ie.

BGP1 and BGP2 are EBGP edge routers.  SW1 is your switch.

1) BGP to OSPF - both routers are redistributing into OSPF and influencing metrics. BGP1 is preferred path for 192.168.1.0/24. BGP1 peering fails but because of the timers the route is still in the BGP table and so is redsitributed into OSPF. So traffic is blackholed. Totally understand that.

2) IBGP between all devices. Same scenario, BGP1 peering fails. It is the preferred path due to local pref. But the route is still in the BGP table and so is advertised to SW1. So SW1 sends to BGP1 and result is the same as 1).

Sorry to keep banging on about this, i just want to make sure i fully understand.

Jon

[John Blakley]

Ah, I understand now....it was my fault for not understanding the question

You're right. Technically, you'd blackhole traffic either way. I guess it comes down to preference? Maybe Rick has more insight on that because I see where you're going now, and I don't think you're missing anything at all.

One other thing that I thought about though with local pref is that it's carried throughout your routing domain across all ibgp peers. Weight doesn't leave the local router, so those two attributes are really the only two preferred methods of influencing outbound traffic. Local-pref would be easier to let all routers know how to get outbound traffic to route vs weight would need to be done on all routers. A little easier on management I suppose.

HTH,
John

*** Please rate all useful posts ***

[Jon Marshall]

John

It just shows that there are so many approaches you can use to achieve the same thing. And i think the way you have done it is another option that i didn't consider ie. in my last post to Rick the options i outlined were -

1) use IBGP and accept traffic will be rerouted from R2 to R1

or

2) use static default routes on SW1 together with IP SLA so that traffic is always sent to the right router and no traffic needs to be rerouted.

It never occured to me to run IBGP on SW1 so it would automatically know using local pref which router to send it to.

I do get stuck sometimes on things that just seem wrong to me eg.

1) rerouting traffic out of the same interface

2) running IBGP on a L3 switch within a LAN

3) having a L3 link between 2 distribution switches which meant HSRP messages have to go via the access layer  switches (made pretty much redundant now with VSS).

Think i just need to be more flexible

Thanks for all your comments, both you and Rick.

Jon

[Richard Burts]

Jon

My thoughts about what makes IBGP better than redistribution is that IBGP allows you to share the policy decisions between your EBGP routers. Lets assume (in reference to one of your previous comments) that you want to select some routes and prefer them on R1 and some other routes are preferred on R2. With IBGP the local preference can be advertised so that policy information is shared between the routers and traffic is routed the way that you prefer. And if the preferred router becomes unavailable then the other router automatically takes over.

It seems to me that in this kind of situation where you want some routes preferred through router A and other routes preferred through router B and you have an interior network then you have choices:

1) inject all of the route detail into the IGP so that all interior routers know precisely for every route whether to forward to A or to B. which means that all interior routers have large tables, lots of updates to process, etc.

2) hide the route detail from the interior, let A and B sort our which is preferred, and acknowledge that sometimes a packet from the interior is forwarded to A which will then be forwarded to B. (redirected)

I recognize that forwarding to A which then forwards the packet to B makes you uncomfortable. And it seems ineffecient. But as the network scales to large sizes, sometimes it is the best way to do things.

HTH

Rick

[Jon Marshall]

Thanks Rick. John, as you have probably read, added a further option when using local pref that makes more sense to me in terms of route selection ie. run it on the L3 core/distribution switch, SW1, in my example.

Like i said in my last post there are just some things i find uncomfortable but that doesn't mean they are wrong to do.

Many thanks for all your comments. Very interesting discussion.

Jon

[John Blakley]

Yes indeed   I like to hear how others are doing it, because frankly you can't get every scenario from a book. You guys are very creative!

[Richard Burts]

I recognize that running IBGP on the switch is also an option. And in some environments (apparently including the one where John works) it makes sense to run IBGP on all the layer 3 devices. But that also suffers from the drawback that it means that all network devices will have all the routes in their routing table and it carries with it the requirements for large table handling, memory, high number of routing updates to process, etc.

One of the things that I believe is apparent in this discussion is that there are many alternatives. And that an alternative that is a good fit in organization A may not be a good fit in organization B.

HTH

Rick