We currently use BGP to peer with our MPLS provider. In one of our areas we have two routers that peer to different provider #
routers. Within our area we run EIGRP and the two routers have all routes for the area. The routers do not use IBGP.
What we are trying to achieve is redundancy so if one of the routers goes down the other router will still advertise all the
area routes. Our provider does not accept the same route from different BGP routers with the same MED so we did the following
router1 - advertise half the networks (h1) with MED of 100
advertise the other half ( h2) with MED of 200
router2 - advertise h2 with a MED of 100
advertise h1 with a MED of 200
Each of the routers redistributes any BGP learned routes into EIGRP.
We are using the "allow-as in 1" command under our BGP config in all our areas.
The problem comes with summary routes. We are using network statements under BGP and so for a summary route we are adding a
route to Null0 into the IP routing table because BGP must find an exact match in the routing table for a network statement.
router bgp 64321
network 10.221.0.0 mask 255.255.128.0
The above network statement is added to both routers.
A static route is then added to closest L3 switch to the router that is advertising the Summary at MED 100.
ip route 10.221.0.0 255.255.128.0 Null0
This summary route is propogated throughout our area bt EIGRP as it seen as directly connected .
what happens is this.
If router1 is the primary router for the summary route it advertises out the summary route to the provider peer.
What i would expect to happen on router2 is:
router2 receives a route for the summary route from the provider peer router. The route will have a weight of 0 as it is
learned from BGP.
( This doesn't matter in terms of routing within the area as there will always be more specific routes within the IP routing
table learned from EIGRP. )
router2 has a network statement for the summary route under it's BGP configuration. It also receives the static route via
EIGRP so it has a matching route for the network statement. So there should be another route for BGP to consider with a
weight of 32768 as it was learned from the IGP routing table.
However a "show ip ro 10.221.0.0 255.255.128.0" shows router2 has chosen the route with a weight of 0 that it received from
it's provider peer.
So what i am unclear about is
1) I thought a network statement added an entry to theBGP table ( as long as there was a corresponding route in the IP
routing table ). Is this incorrect ?
2) Obviously because we have "allow-as in 1" in our configs the routes advertised out from router1 to the provider router
will come back in via router2 and presumably be redistributed into EIGRP. Is this going to cause a major problem
3) if i do a "show ip bgp neigh x.x.x.x advertised routes" on router 2 it does not show it advertising the summary route.
Is this because it is receiving the route from the provider router and so won;t advertise it out ?
Any help would be much appreciated.
It is a matter of AD. Let's assume that the EBGP connection between router 2 and the PE is not up. EIGRP will have a route to 10.221.0.0/255.255.128.0 which will be installed in the routing table. Even if the events occur in the opposite order, the same thing will result.
Now, the BGP session comes up and 10.221.0.0/255.255.128.0 is now learned via EBGP. EBGP has a better AD than EIGRP so it will displace the EIGRP route in the routing table. The EIGRP route will not appear in the BGP table because it is not an active route, even though there is an exact match in the routing table. BGP only installs active routes in the BGP table.
The way to solve this is to include the following command under your BGP config on router 2:
network 10.221.0.0 mask 255.255.128.0 backdoor
The route will then be given an AD of 200 and your EIGRP route will be preferred. It will then be installed in the BGP table and advertised as the best BGP route.
Hope that helps - pls rate the post if it does.
I appreciate your response but this is where i am getting confused. In my previous test lab i had two sites. Each site
had a router connected to a PE router ie.
router1 to PE1
router2 to PE2
The sites were connected to each other via a backup link but all traffic between sites should go via MPLS
in normal circumstances. I set it up as follows:-
router1 advertised its subnets ( r1 ) with MED of 100 and the subnets of site 2 (r2) with a MED of 200
router2 advertised its subnets ( r2 ) with MED of 100 and other sites subnets (r1) with MED of 200.
Each router had network statements for each of the advertised routes under its BGP config.
BGP routes were being redistributed into EIGRP in each site and so had AD of 170. To stop the internal routers
in each site forwarding traffic over the backup link ( because these routes would be EIGRP AD 90 ) I made the
point to point interfaces on the backup link passive under EIGRP. On each backup router we added static routes
for the other sites subnets pointing to other end of the p2p link and redsitributed these into EIGRP with an
So now we had routes for the other site coming in via BGP and being redistributed into EIGRP and also being
redistributed from the backup router into EIGRP - both with an AD of 170 ie
router1 receives routes from PE1 for Site 2's subnets and redistributes into EIGRP.
The backup router in Site1 has static route entries pointing to the other side of the backup link and is
redistributing into EIGRP.
What i found on router1 was that the BGP table showed two routes for each subnet. One had a weight of 0, the
one received from PE1 and one with a weight of 32768, the one presumably in the BGP table because of the
network statement. The summary routes were installed, i assume, because the backup router was redistributing
them into EIGRP so router1 found a corresponding entry in the IP routing table.
The only way i could get router1 to send traffic out via MPLS was to add a weight to the route received
from the PE1 router. I added a route-map to the config to adjust the weight of all routes received from
PE1 to 40000. Traffic then went via MPLS. Without it router1 sent all traffic via the backup link.
We implemented this design at some of our paired sites and it has worked in failed link situations.
Hence the reason i'm confused. I guess my confusion is based on the "network" statement. Following on from your
explanation if the connection between router1 and PE1 is down don't you still get an entry in the BGP table
because there is a network statment and a corresponding route.
Many thanks for your help so far - hope this makes sense.
Wow Jon... you've got some setup here !
Firstly, could you clear up some confusion I have: in your first post, I understood that the problem was that all traffic between the 2 routers at your site was going through the MPLS cloud and not directly to each other. Is that correct ?
In your second post, the situation you describe seems to be the opposite - you want the traffic between r1 and r2 and not through the MPLS cloud but that is not happening. Is that correct.
Assuming the above is correct, the solution I proposed to your first problem still stands. Have you tried it out ? I'd like to know how you went ...
Now, looking at your second issue...
The reason you are not following the BGP routes is because the static routes you have created squash everything else due to their better AD.
I believe there is a simple fix - on each backup router, configure the static route for the other site with an AD of 250 i.e on r1:
ip route r2.x.x.x y.y.y.y
This will result in the following:
- when the BGP link is down, this static route will be used to get to the other site via the backup link.
- when the BGP link is up, the BGP route (with an AD of 20) will be installed in preference to the static route. Therefore, the static route will not be injected into BGP. So how do you get redundancy here ? Okay, if the BGP link to R2 goes down, router R2 will no longer be injecting the route for R2 into BGP. Therefore, it will also be withdrawn from the update to R1. When that happens, R1 will install its static route into the routing table and redistribute it into BGP now. The PE will learn this route via R1 now and let the rest of the CEs now... So you've got your redundancy...
Trust me... that will work...Just because you don't see multiple routes in the BGP table when everything is active does not mean that you will not get redundancy. Try and follow my reasoning above.
Hope that helps - pls rate the posts if it does.
Many thanks for all your help. I intend to try the backdoor command in the lab at work when i test it today.
The situations i described are different as you say. in the first post i want traffic not to go via MPLS but to go
across the internal network (it's a MAN with Gig LES links). Traffic does go internally because of the more specific
routes but my concern was that I thought i would see both routers advretising out their and the other routers networks
but i don't. I see router1 advertsing r1 subnets (MED 100) and router2 r2 subnets (MED 100).
In the second situation i do want traffic to go via MPLS because it is a quicker link than the backup link. one detail
i forgot to mention which you have spotted is that we do have an AD of 254 attached to the static routes on the
backup routers. We applied this to stop the backup routers themselves using the backup link.
The one thing i still don't understand is this. Once those static routes have been redistributed into EIGRP they get
a value of 170. The backup router is not the CE router. The CE routers in my post are router1 and router2. So if
router1 receives a redistributed static from it backup router the AD is not better. Router1 gets a route for it's
partner site via BGP with AD of 20 and a route from it's backup router with AD of 170. On router1 there is a network
statement for this route as well. Because of the redistributed static router1 finds this route in the IP routing
table and so installs the route in its BGP table. So now i have two routes in the BGP table - one with a weight of
32768 ( the network statement ) and one with 0 received from it's PE both with an AD of 20. So it chooses the 32768
route via the backup link which is why i have to manipulate the weight. I'm going to lab this up again just to
confirm that this happens. Like i say, it's been tried and tested in production. I'm also going to read up on BGP
again because i think some of the confusion is down to my understanding of what the "network" statement is actually
doing to the BGP table.
I think you have pretty much answered my question though. I believe there is redundancy in both scenarios altho
short of taking out a production link it's a bit difficult to test ! and i quite like my job :-).
Thanks for clarifying this...
I was assuming that the CE routers and the backup routers were the one and the same.
Now let me explain what is happening in yourlab setup (now that I have the full picture).
Once again, let's break this down into a sequence of events:
- assume the BGP link is not up yet
- the EIGRP external routes are learned by the CE router from the backup router and are given an AD of 170. This route is then installed in the routing table. Since the route is active and has a corresponding network statement under BGP, it will be installed in the BGP table. However, since there are no BGP sessions, this does not do much.
- now the BGP session comes up and the router learns the same backup route via EBGP
- since this is a BGP-learned route it will also go into the BGP table
- the BGP decision process will now run and select the locally injected one, since it has a higher weight (32768)
- since the local route is now the best router, the CE router will advertise this as the best route to the PE
- when the PE receives that route from the CE, it will now have 2 BGP paths for the same route so it will run the decision process and guess what ... it will pick the one learned from this CE router, since it was learned via EBGP. This route will now be installed in its routing table
- when it does that, BGP on the PE will realise that the first route it advertised to the CE is no longer valid so it will send an UPDATE msg to the CE withdrawing the route
- this will result in the CE having only a single route in its BGP table, the local route
Now, does that match what you see in your lab setup ? With this setup, does 'sh ip bgp' on the CE show only one route, the local one ?
When you manipulate the weights, the local route is no longer preferable and this situation does not happen.
The key thing to understand here is that BGP only advertises active routes i.e routes that are installed in the routing table. The other thing is that whileyou think the AD should be a determining factor, the BGP AD does not come into the picture until the decision process is done - in fact, an IBGP path (AD of 200) will easily beat an EBGP path (with AD of 20) if it has a better weight or local-preference or AS-PATh lenth or origin...
Hope that helps.
A bit more info on a point I mentioned in the last post but which deserves explanation - BGP route withdrawal.
Say you have a setup such as: RTA --- RTB --- RTC
Now, imagine that A runs BGP with B and B runs BGP with C. Suppose C advertises a route R to B. B will then advertise that to A and all is cool. Some time later, A learns this same route (via some other means..maybe an IGP). A installs this route as the best BGP route. As a result it advertises that route to B. Now, the attributes of the route are such that B also now believes that that is the best route and installs it in its routing table. At this point, B will send a BGP UPDATE message to A expliclitly withdrawing the original route it sent (since it is no longer active in B's routing table).... This is a very crucial thing to understand and you are seeing just that in your setup...
Hope that helps - pls do rate posts that do...
Okay, that now makes a lot more sense ( I should have explained the setup in more detail in the first place). I can't
get to the lab at the moment ( working from home and link problems at work ) but i when i do i'll check what
"sh ip bgp" does show.
i do need to get this setup again in the lab because, if i understand correctly, you are saying without the weights
the following happens ( assuming BGP connections are up )
CE1 (Site1) advertises route 10.170.1.0/24 (which exists in Site2) out to PE1
PE1 has two routes for this route (one from CE1 and one from other PE router connected to CE2) and chooses CE1 advertised
PE1 will then advertise this to all other PE routers.
So in effect the rest of our network thinks the best way to get to 10.170.1.0/24 is via CE1 which under normal
operations (ie no failed links) is exactly what we don't want to happen because the subnet actually exists in Site2.
I manipulated the weights purely to stop the CE1 router choosing the local route and routing all traffic via the
backup link. I never took into account how this affected advertisements back to the PE routers.
This is why i need to lab it up again because i don't recall that happening but what you say makes perfect sense.
Obviously we do have redundancy because if CE2's link fails then CE1 will then choose it's locally defined route.
One last question, if you don't mind. We have no choice but to use BGP as this is what our provider wants, and we
do need this sort of redundancy. Is there a better way to achieve the redundancy i'm looking for - better in terms
of simpler really or is this about the best i can do. ?
Once again many thanks for all your help
I think your config is more complex because of the redistribution that you are doing. Configuring iBGP and influencing the right BGP attributes should make the situation better and easier. As I'm a little lost with respect to the topology and requirements, I cannot give you a sample config.
Actually, after I had posted that last message and gone to bed, I realized that I had forgotten something... the MEDs you had assigned ...doh !
That means that your PE will choose the right router as the best path for the route, not the backup router. So that means you should see 2 routes in your CE BGP table... My apologies for that ... it's just a bit hard to visualise these things without the routers in front of you.
However, my explanation was correct for the case where the MEDs were equal...