Dual HQ Links to Service Provider MPLS - Need TE?

Unanswered Question
Dec 19th, 2006
User Badges:
  • Bronze, 100 points or more

We are switching from an Internet VPN (GRE tunnels) WAN to the Sprint MPLS network. We are using EIGRP across it, as this was the simplest thing for the conversion. We have an OC-3 link at the Data Center on a 7606, and various T1/NxT1-PPP/DS3 links at the remote sites.

We are adding a second OC-3, using a second 7606. The initial turn up shows that the routes we receive on it have (mostly) identical metrics (the OC-3's are on separate PEs but in the same MPLS POP).

While we want to use both OC-3s (we will need to soon), we do NOT want to "load balance" (i.e. automatically) across them, especially not the VoIP traffic.

What we would like to do is, "direct" Group A remotes to come in via PE1 and Group B via PE2. We will periodically adjust membership in the Groups to roughly balance the load.

The SP has suggested SOO as a way to do this, but I'm not sure that's correct. I *do* think we will need SOO at some point, we are going to have a "backdoor" link to our DR site 1Q07, but I don't think SOO will do wat we want.

I looked around a bit and see that I can "set extcommunity rt" in a route map (found while looking at SOO), but my knowledge of MPLS and BGP is limited (the SP does use BGP in the MPLS WAN, of course). But "route target" surely sounds like it may apply.

Can someone give me a brief explanation of route target(ing) and/or any other suggestion by which we can accomplish this. The SP will not support multiple VRFs (I don't think) nor multiple EIGRP instances.

Thanks for the help!


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 3.7 (3 ratings)
dknov Tue, 12/19/2006 - 16:56
User Badges:


Are you still using EIGRP as a PE-CE protocol or have you switched to eBGP?

If you are still using EIGRP, then you would really need Sprint's help in it. Let's say you have remote site A. On PE router where this remote site connects, Sprint will redistribute EIGRP into their iBGP (and will assign RT, but you do not need to worry about it). When this iBGP advertisement reaches PE router where your 7609 is connected it will be once again redistributed to EIGRP (and sent to 7609).

The EIGRP metric you would see on 7609 will be determined by what metric is set on the last PE during iBGP-to-EIGRP redistribution. This is something you cannot control and you would have to cooperate with Sprint, so they use route-maps during redistribution to asign different matrics to different prefixes.

If you are using eBGP as your PE-CE protocol, you can prepend your AS on eBGP connection at remote site. This AS_PATH attribute will travel through Sprint's iBGP session and eventually end up in your 7609s.

By manipulating AS_PATH attribute at source (remote site) you can manipulate which outbound link is used in your headquarters (7609).

Just as a comment, RT is an extended community attribute carried in BGP VPNv4 NLRIs. Even if you run eBGP as your PE-CE protocol unless you follow CsC (Carrier Supporting Carrier) architecture, this is not something which is useful to you. It might also be applied in Multi-VRF (VRF-lite) environment, but you mentioned that Sprint won't support that....

I hope I was helpful.


PAUL TRIVINO Tue, 12/19/2006 - 22:01
User Badges:
  • Bronze, 100 points or more

David, thank you, yes it's helpful. I wasn't specific enough, I do know I need their help but they haven't been able to come up with a way for us to do this!

Yes, we're still EIGRP PE-CE. I guess I was hoping for something we can set in EIGRP that will get used in their BGP (I am trying to provide them with "use this, fellas" since they haven't been able to say "we'll use this, Paul." ;^)

We have (and are still) considered switching to BGP but it likely means switching all sites to BGP, we don't think we can do it with just the Data Center 7606's. My current knowledge of BGP wouldn't fit on the head of a pin but a 4d nail would have room. And although I'd love to learn, the conversion itself would be a PITA.

Any other ideas? TIA.

dknov Fri, 12/22/2006 - 10:30
User Badges:


If having Sprint cooperate is not an option, the only other thing that comes to my mind is that you can use EIGRP's option of modifying the metric in the inbound direction.

This can be done in both your 7609s and on remote sites.

Use offset-list under your EIGRP process to change the metric of inbound advertisements you get from the PE. By doing so, you could manipulate which of the PE-CE circuits you're using for this particular prefix.


PAUL TRIVINO Fri, 12/22/2006 - 12:38
User Badges:
  • Bronze, 100 points or more

I haven't looked at this yet, since I just got the 2nd line operating Monday, but it occurs to me that each remote has only one PE and so only one next-hop. I could certainly use one OC3 or the other at the HQ end by adjusting the remote net's metrics, but I don't see how I could send the remote site traffic BACK via a specific OC3 PE.

FYI Sprint isn't uncooperative they just haven't suggested a method.

Thanks for your input.


hkezaby.mpls Sat, 12/23/2006 - 20:53
User Badges:

hello , i have an important question which will create a scenario in my mind based on your answer.

Do you want to separate the remote sites traffic based on site criteria no matter what type of traffic it is ? i.e for example you have 10 sites , you will separate 5 sites with all their traffic to one of the hq 7606 and the other 5 to the other.

Or you want to direct certain type of traffic for example voice to one of the 7606 and other types for example data to the other i.e the 10 sites will have access to both 7600's based on the type of traffic.

Another question : is redundancy to both 7600's links an important design issue for you i.e if for example five sites or certain traffic is accessing one of the oc3 links in the hq and this link fails , do you want all the traffic to be directed to the other oc3? is that what you agreed with sprint upon ? or you will have some loss of connectivity for some sites or traffic until link is recovered ?

Thank you ...

PAUL TRIVINO Sun, 12/24/2006 - 11:59
User Badges:
  • Bronze, 100 points or more

Absolutely yes to both questions.

It won't be a division by number of sites, we'll use traffic load to assign a remote to one OC-3 or the other, but yes all traffic from any particular site will use a single OC-3.

And yes, we absolutely must have the redundancy, with one possible change being this: we will have a large (OC-12) link from HQ to the DR site, in 1Q07. We might be willing to have both "sets" of sites use the DR site as their "backup" link, such that the failure of either HQ OC-3 sends their traffic to the DR and then back to HQ via the OC-12. The DR site will have its own OC-3 link to the MPLS cloud as well as the backhaul link to HQ.

Thanx for your interest, looking forward to hearing from you.


hkezaby.mpls Mon, 12/25/2006 - 02:14
User Badges:

OK this is perfect ...

what i'm suggesting you do is to tell sprint to assign a bunch of sites (decided by you) an RT extcommunity and the other bunch a different RT.After that they will import in each one of the HQ PE'a a different RT than the other which will cause separate sites routing information to be redistributed into EIGRP process of the PE and by then to your devices.

If we keep the solution up to this only we won't have any redundancy so what you should do next is to let sprint import both RT's for all the set of sites into the DR PE (or they could assign a third common RT for all the remote sites to be imported in the DR PE ONLY).

please tell me if the oc-12 link is a dedicated leased line (dark fiber for example) or a layer3 mpls shared link.

In the first case sprint will only have to increase the metric (make them less desirable) of redistributed routes from the MPBGP process to the EIGRP inorder to choose those routing information as the last option when one of the oc-3's fail and not to choose it whenever both HQ links are active.You will not have any problems with external vs internal eigrp learned routes because all those routes already reached the DR PE as external and this is the way they will reach the HQ through the oc-12.

Waiting for your response ...

PAUL TRIVINO Mon, 12/25/2006 - 09:49
User Badges:
  • Bronze, 100 points or more

This sounds pretty good so far. The HQ-DR OC-12 is a completely separate link, dedicated leased line.

As an aside, I had found the route-map "set extcommunity rt" facility and asked Sprint if that would work (not knowing how it works) - I haven't had a (sensible) answer yet. They suggested SOO - which I *will* need to reduced problems from the backhaul OC-12 - but SOO will not solve THIS OC-3 assignment problem, I'm sure.

So, if Sprint will set up the rt targets (right?) in the MPLS cloud, can we use "set extcommunity rt..." in the remote CE routers, to control what goes where ourselves? I say this because Sprint is fairly responsive to up/down/error issues but the more complex things take a lot longer to address. If we can have them set up the system once and we do the ongoing maintenance, we can monitor the load and adjust it ourselves.

HTH, thanx.


hkezaby.mpls Tue, 12/26/2006 - 15:08
User Badges:

OK "set extcommunity rt" might work if you are already exchanging extended communities with sprint which i think is not happening because extended communities are only exchanged via BGP updates (as long as i know) so the only routing protocol that might be used in that case is BGP.

In your case and since your PE-CE routing protocol is EIGRP , setting the RT's could only be done inside sprint cloud.That's why i suggested my last post solutions.

I prefer if you suggest it to them and see what they might say.

I'm totally supportive with you when you say you want to remove as much administrative burden as possible upon them inorder to have a quick suitable setup but in all cases since you are using layer 3 mpls links , you will always need their help especially in routing decisions with redundant links.

One more idea i have which will make things much easier for them (but will never negate their important role) is to use ordinary route tagging with route maps on your CE's (done by you).You could simply tag all the routes coming out of a group of sites with a certain tag lets say 19 and the other routes from the other set of sites with 29 and let sprint match with another route map on their HQ-PE (done by them) those routes and change their metric on the HQ PE's while redistributing them into EIGRP.So that tags 19 take a less metric (preferred) on HQ-PE-1 which is connected to HQ-CE-1 and other routes with other tags take default metric (less preferred) and vice versa on the other HQ-PE.By this you will have redundancy and separated sites on each HQ-PE.

Please inform me if that's ok with you and THEM :-)


This Discussion