We are in the process of creating a WAN Distribution layer. Prior to implementation, we would redistribute BGP routes learned from our different MPLS carriers into one core EIGRP instance. Our WAN consists of primarily two different routers. One one router, we will call R1, there is a Provider A. On router 2, R2, we have two other providers, Provider B and Provider C.
Provider A and B give us our primary circuits in our remote locations, which typically consist of T1 circuits. Provider C is a DSL provider and allow for a backup connection that is always on. As I said before, all of the Providers are BGP MPLS connections.
Here is the current topology in graphical form:
Problem: Suboptimal Routing for traffic because of the common redistribution metric that is used between the access routers and the WAN distribution switch.
Scenario #1: For remote location A, the primary circuit is comes from Provider A, and the backup circuit comes from Provider C. Provider A has QoS enabled on the MPLS network, carries voice traffic and is a larger circuit with stricter SLA's. Provider C is strictly a data only connection without any QoS and is not designed at this time to carry voice. Also the links from Provider C are typically smaller. So there would be a drastic performance hit both from the applications and voice perspective. Once the routes are redistributed into EIGRP, both paths become equally in cost and therefore equal cost load balancing will occur when traffic comes from the core through the WAN Distribution switch, with some of the traffic going over the preferred path, over Provider A, and some of the traffic going over the path through Provider C.
Scenario#2: For remote location B, the primary circuit comes from Provider B, and again he backup circuit comes from Provider C.
Solution #1: Rather than distribute routes from BGP to EIGRP on the access routers, create IBGP neighbors between the Access routers and the WAn Distribution switch and use local preference to control the routing for Scenario#1. For Scenario#2, use weighting to prefer Provider B over C.
Solution#2: Rather than create IBGP peers, prepend the Provider's C AS using the AS path prepend, and then adjust the redistribution metrics between EIGRP and BGP on the access routers so that the WAN Distribution will see the preference for the primary circuit to go over Provider A or B.
So which solution would you choose? Why?
Do you have any other ideas?
You have two things to fix:
1- R2 needs to prefer Provider B to join site B -> the weight attribute is there for that so I would use it
2- Wan Distribution switch needs to prefer R1 to join Site A: I would configure R2 to set a BGP community to any routes received from provider C and when you redistribute your BGP routes into EIGRP, set a higher metric based on the community.
I would migrate to BGP if you have host/routers connected to R2 because in this case R2 needs to go via the Distribution router to join Site A. It will be easier to play with Local Pref than with Administrative Distance between EIGRP and BGP.
Could you explain in a little more detail, maybe by way of example, what you meant by;
"It will be easier to play with Local Pref than with Administrative Distance between EIGRP and BGP."
If R2 receives some traffic destinated to Site A, it will have the choice between a BGP route and an EIGRP route. By default eBGP routes have an Administrative Distance (or AD) lower than EIGRP ones so R2 will always prefer its BGP routes which is not what you want in your case. That's why you will have to play with AD so the one for EIGRP will become lower than the BGP ones (or the opposite).
If you are running BGP instead of EIGRP, R2 will have the choice between an iBGP and an eBGP route so playing with AS length or Locap-Pref will allow you to make R2 chosing its iBGP route first.
First let me first say thank you for you insight.
We typically don't have a traffic flow that would flow betwen Site A and Site B. The most of the traffic would be coming from the Core. If for some reason there was a communication requirement that needed to go between the two remote sites the way that I see that traffic flow is such:
Packet would leave Site A and be forwarded to R1 where it then have only the choice of a BGP route that learned from the WAN Distribution switch. The WAN Distribution switch would receive the packet and forward it to R2 and the packet would route over the preferred link, that being through Provider B.
Packet would leave Site A and be forwarded to R1 where it then have only the choice of a EIGRP route that learned from the WAN Distribution switch. The WAN Distribution switch would receive the packet and forward it to R2 and the packet would route over the preferred link, that being through Provider B. This is due to the BGP weighting that takes place to prefer R2.
I know that both solutions would work. I am more looking for what solution seems to make more sense and but more importantly why/why not. I am looking for a consensus amongst all in the support community as two what they would prefer between the two different solutions, or any other suggested solutions that might make sense.
I agree this path is fine. This is the return path (site-B to site-A) I was thinking about. In this case, R2 will need to choose between the eBGP route learned from Provider-C and a route learned from R1 via iBGP or from the Distribution Switch via EIGRP. So the way to make R2 chosing R1 to join site-a depends on the protocol you are running (BGP or EIGRP).
You should go with the solution you (and your team) are most comfortable with in term of support and troubleshooting as both of them are supported and works fine.
Personnaly, I would go with BGP as it allows you the finest control for your routing policy and because I'm comfortable with it. But you would need EIGRP anyway so R1 and R2 can join each other.
If you go with iBGP, the prefixes sent by R1 will be installed by R2 with R1 as the next-hop (and vice-versa) so the BGP route is not enough as there is a level of recursivity. To resolve it you need another route provided by your IGP to know how to reach the BGP next-hop.
But if we only send a default route from the WAN Distribution, we wouldn't have to form iBGP peers between R1 and R2, correct?
If that is so, then there is no need to use an IGP, correct?
If you want to use BGP internally, you have two choices:
- Fully-mesh iBGP session between your 3 routers
- Use your Distribution router as a Route-Reflector so you would not need to build an iBGP session between R1 and R2.
In any case BGP updates sent by R1 and received by R2 will have R1 ip address as the NH. As they are not directly connected, R2 needs to know how to reach R1 ip address.
The following link is a short introduction to BGP:
Now without BGP, if R2 receive only a default-route from the Distribution router, it will never use it to send traffic received from Site-B and destinated to Site-A as it has a more specific one received from Provider C so you should be careful if Inter-site traffic will become a reality in the future.
If you are not familiar with BGP, you should stay with EIGRP as you have a working solution.
I think I am going to far in my thinking than I need to. I understand BGP enough to implement the original solution, just wanted more or less to get confirmation. I missed the Full Mesh iBGP so it was nice to have you add that to the design.
As a next step, and more from optimization as well as to create a centralized routing point, I was thinking that if all I only allowed the default route from the WAN Distribution to R1 and R2, then R2 would never learn the routes from R1. So if there was a need for intrasite communication, the packet would leave Remote 1->R1->Dist->R2->Remote 2. I would probably create an internal peer group, create a filter to only allow 0.0.0.0 0.0.0.0 out that it receives from the redistribution between the core EIGRP instance into BGP.
Do this makes sense?
Does it removed the need for a Full mesh iBGP?
Would this go against any Cisco Design recommendations?
If you don't want a fully-mesh iBGP session, you must use RR function.
Based on your design, it makes sense to have the Distribution Router (let's call it DR, faster to write ;-) ) configured as a BGP RR so you just need iBGP sessions between him and R1 and R2.
You focus only for traffic from Site-A to Site-B but the return path will not be symetrical if R2 has the choice between a default-route from the DR and a specific route from Provider C. The last one will be chosen as the rule is always to select to longest prefix mask.
Your right, I didn't think about the return traffic.
So I think what I want to do is the following.
Configure Fully mesh iBGP between DR, R1 and R2.
Configure an internal peer group on DR to only allow the default route down to R1 and R2.
Redistribute BGP into EIGRP at the DR back into the Core.
On R1 configure a local preference for Provider A.
On R2 configure a weight to prefer Provider B.
It seems as though this will allow me
1. To prevent the suboptimal routing.
2. Provide a central routing point that makes it easier to control routing.
Did I miss something?
I wouldn't normally post especially at this point of a thread, but since you asked at an ealier point about consensus amongst all in the support community, I will only say that I think Laurent is being extremely helpful here.
Almost perfect ;-).
You still have the issue if Site-B wants to talk to site-A, this traffic will never go via the DR as R2 has a more specific routes coming from Provider C.
So in addition to the default route, I would let Site-A and B subnets being propagated too via BGP. As R1 is configured to set a higher LP for its eBGP route, DR will prefer it as primary path. R2 is the only one announcing Site-B subnets but due to its weight configuration, it will prefer Provider B as primary. R2 will also receive Site-A subnets as iBGP routes with a higher LP so it will choose this one as primary.
This is the conclusion that I came to as well. I put all of the points that we have discussed here into an emulator and discover exactly why you said would happen, that is the LP would take precedence over the default route. I configured an internal peer group on the DR, made the DR a route reflector, and the routing works as intended. Thanks for all of your help, I hope to work with you again in the future.