Brand new MPLS, how to configure CE routers?

Unanswered Question
Jan 26th, 2012

Hello all,

Please congratulate me, and here, accept a cigar.   As I'm a proud new owner of an MPLS network.  As with any new dad with their first born, there's nothing but endless questions and over analyzing everything.  I'm no different with this MPLS network.  I need someone to clear my head, and tell me it’s going to be okay!  Really, I'm sure new borns are harder than MPLS!   

Everything I read on the internet, tells me information about MPLS, from the point of view of a provider.  What’s in the provider’s cloud.  How it works.  What MPLS services/applications exist and how to use them.  Although all this information is great, and is good reading.  I have been looking for information from a customer only (CE) router configuration point of view.  A checklist of things I need to know from the provider, things that need to be configured on my CE routers, and/or the configuration of the PE-CE link.

This is where I stand today.  We currently have a network that uses IPsec VPN’s to connect remote sites to a hub network.  To improve services and a host of other objectives we are moving to an MPLS solution.  Most of the network, mainly the remote sites will be connecting over T1 circuits, and one of the two hubs resides in a CoLo and will have an Ethernet handoff.   What has been done so far (and before me joining the team) was the provisioning of these circuits with a provider, and purchase of the routers.  So in other words, a completely brand new MPLS network needs to be configured.

So here are some of my questions thus far (some questions are specific to this project, others are just for self education, like, how most of the world does it):

  1. I assume the most used MPLS application is MPLS VPN?
  2. Normally when customers go to providers for MPLS services, when is the type of service to be used talked about?  E.g.  To use MPLS VPN or to use VPLS, or something else?  Currently, I have nothing from my provider telling me what we are getting.  The only thing I see as for “Product” = “DS1 Private Port”.
  3. Along with the Product information from the provider, the only other info I have at the moment, is serial IP’s.   One being the “customer serial IP” which looks like an internet IP address. 
    1. Is this IP for the CE router or the PE router?
    2. If it is for the CE router.  Then I’m lost.  How does that work?  I would assume that my serial interface on my CE router would have a Private IP that we configure?  A link for how this configuration would/should look like would be great.
  4. What MPLS services require that the provider and the customer work together on configuration tasks?  (Sorry, if this sound noobish)
  5. In MPLS VPN, is the configuration of CE routers completely independent of the provider’s knowledge?  Because I read things like this, “In MPLS VPN, PE routers participate in customer routing, providing optimum routing between sites and easy provisioning of sites.” From this link.  And I get all confused! 
  6. I’m really at lost with this one.  If we wanted to used QoS and/or/with MPLS Traffic Engineering?  Does the provider and the customer have to work together to share some information to make this work?  For example, I think question 5 is the case, but I can be wrong.
  7. I have heard that some providers block dynamic routing updates, is this true? If so, WHY?  It seems if MPLS VPN = kinda like a Private PtP, then why block anything?
  8. I’m assuming we are getting a MPLS VPN solution provided to us.  Does this normally come fully meshed?  Or do we dictate this or Hub/spoke to the provider?
    1. Or do we configure topology via routing?

As you may have figured it out, there is not much coming from our provider at the moment.  To give them an excuse, most circuits are not even fully installed at the moment.  But I would have just assumed a conversation took place that would answer most of these questions.

Thanks for your time and help,

Nick

I have this problem too.
1 vote
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4.5 (8 ratings)
rsimoni Thu, 01/26/2012 - 23:28

Hi Nick,

let me try to answer to your questions:

1. Yes. EoMPLS and VPLS are picking up but MPLS VPB is still the most widely MPLS application being used.

2. Cannot answer as DS1 seems some naming convention used by your ISP. You need to ask them.

3. I think it is the public IP addressed assigned to the WAN interface of the CE. It might also be the PE address, but this does not change a lot as CE and PE addresses are, of course, on the same subnet.

Some providers use pricate addresses for CE-PE connection while others (bigger ones) can use public addressing. It is just their choice which does not interfere with your VPN.

4. For scenarios where CEs are owned by the ISP the latter takes the entire configuration responsability, meaning that they should know your design, your address scheme etc.

For cases where the CE is owned by the customer (I think your case) you only have to agree on the routing peering between CEs and PEs (usually BGP) but then the customer advertise anything he likes. The ISP does not care about your VPN routing in this case.

5. ISP necessarily partecipates in your routing as they manage the MPLS backbone that is needed for the VPN to work. The make sure that traffic exiting from a CE (for them entering a PE) is routed to the correct egress PE withing their cloud. They are not aware of your routing but they need to provide correct routing for your VPN traffic.

6. Actually in this case customer don't do nothing. Usually you seat on a table with your ISP and you give them your requirements in terms of redundncy and in terms of quality of services you expect. Then they implementation the most suitable solution for you (completely transparent to you) on their devices according to the existing network capabilities.

7. AFAIK ISPs don't interfere with customer routing... not sure if this is referring to some specific caes or scenarios I am not aware of. If you manage the CE you have total control of what you advertise.

8. Normally MPLS VPN comes in Any-to-Any model which from a logical point of view is (almost) equivalent to a full meshed topology. If you want a hub and spoke topology you need to ask your ISP and they will configure for you (transparent configuration for you).

No topology configured from customer.

Hope it helps you somehow

Riccardo

nickhesson Mon, 01/30/2012 - 18:54

First off Riccardo, I thank you for the quick reply.  But i need a little more clarity (me being over analysing maybe)

4. For scenarios where CEs are owned by the ISP the latter takes the entire configuration responsability, meaning that they should know your design, your address scheme etc.

For cases where the CE is owned by the customer (I think your case) you only have to agree on the routing peering between CEs and PEs (usually BGP) but then the customer advertise anything he likes. The ISP does not care about your VPN routing in this case.

You are correct, we own the CE routers that I must configure.

Although this information DID help, it really didn't answer my question.  Morely, because I think the question was worded wrong.  I guess my overall question is, It seems as if the ISP plays a role in your routing for MPLS VPN services, are the other MPLS services handled in the same manner?  Meaning that when companies deploy MPLS, they are really teaming up with ISP's to deploy them?  Not like Frame Relay, where they worked on the frame relay switches, give you hand-offs, and the customers would configure their routers have to implement everything else.  e.g. Routing

5. ISP necessarily partecipates in your routing as they manage the MPLS backbone that is needed for the VPN to work. The make sure that traffic exiting from a CE (for them entering a PE) is routed to the correct egress PE withing their cloud. They are not aware of your routing but they need to provide correct routing for your VPN traffic.

???  Sorry, but you lost me on this one.  Does this mean i will be using two routing instances/protocols?  One for the PE-CE link to update the ISP vrf's and the other to manage our internal routing table?  (forgive me, still trying to graps everything.)  Or use one, and the ISP will handle the rest?

Again thanks for your time and help,

Nick

Peter Paluch Tue, 01/31/2012 - 16:08

Hello Nick,

I am taking the liberty of answering as Riccardo indicated he's overloaded this week. Still, I would like very much Riccardo to at least double-check what I put here.

I guess my overall question is, It seems as if the ISP plays a role in  your routing for MPLS VPN services, are the other MPLS services handled  in the same manner?  Meaning that when companies deploy MPLS, they are  really teaming up with ISP's to deploy them?

I am not entirely sure if I understand where you're heading. However, you should note that the MPLS itself is not usually extended to the customer (it can be, but for simplicity, ignore that for now). That means that the customer does not use MPLS labeling, does not participate in label assignment protocols, etc. - in other words, the MPLS does not exist for the customer. There may be some kind of coordination between the customer and the SP, but this coordination is related to something different than MPLS alone - for example, common IP addressing on the PE/CE link, common L2 encapsulation, common routing protocol, etc. - these areas are completely oblivious to MPLS.

???  Sorry, but you lost me on this one.  Does this mean i will be using  two routing instances/protocols?  One for the PE-CE link to update the  ISP vrf's and the other to manage our internal routing table?  (forgive  me, still trying to graps everything.)  Or use one, and the ISP will  handle the rest?

What Riccardo was trying to tell you is that the ISP collects the routes reachable at every customer's site (thanks to the PE/CE routing protocol) and then distributes this information - about the customer's networks located at particular sites - to the other PE routers, in order to allow any customer's site to talk to each other. When a customer's site sends a packet to another site, the ISP's network has two issues to solve:

  • Where (behind which PE) is the customer's destination network located?
  • What is the shortest route to that PE?

Here, the ISP's routing very much coincides with your routing. You advertise your networks to the ISP. ISP disseminates this information among his PEs and when it is routing packets between your sites, it uses both your information to find out if the destination network exists at all and what is the egress PE, and its own routing information to reach the PE via the shortest route available.

On a site, you simply run a single routing protocol. It's as easy as that.

Best regards,

Peter

nickhesson Tue, 01/31/2012 - 16:55

Hey Peter,

Thanks for taking the time to answer my questions, I really appreciate all the support guys. 

When you are first reading about what MPLS VPN is.  You hear a lot of its like or similar to Frame Relay or ATM.  Now I have never had to work on ATM other then a lab in GNS3.  But Frame Relay yes.  And you as the customer of the Frame Relay, get to choose everything, serial IP's and routing protocol.  But in MPLS that is not the case.  The ISP dictates this for you, ok ok, the SP and you work together for a common IGP.

But in the Frame relay, you have control or monitor abilities over all routing neighbors.  Meaning that your next hop router is your router.  Not a router owned and operated by the ISP.  In this MPLS VPN, you don't have that same separation.  You are now relying on the ISP more now.  Not just for L2 services but L3 as well. 

It was hard at first to get my head around this, but now.  I'm Game! 

I guess my overall question is, It seems as if the ISP plays a role in  your routing for MPLS VPN services, are the other MPLS services handled  in the same manner?  Meaning that when companies deploy MPLS, they are  really teaming up with ISP's to deploy them?

I am not entirely sure if I understand where you're heading. However, you should note that the MPLS itself is not usually extended to the customer (it can be, but for simplicity, ignore that for now). That means that the customer does not use MPLS labeling, does not participate in label assignment protocols, etc. - in other words, the MPLS does not exist for the customer. There may be some kind of coordination between the customer and the SP, but this coordination is related to something different than MPLS alone - for example, common IP addressing on the PE/CE link, common L2 encapsulation, common routing protocol, etc. - these areas are completely oblivious to MPLS.

I guess i'm asking does the ISP play a bigger part in your network when rolling out other services, other then VPN? 

Again, guys you cleared my head a lot.  Thanks very much.

Nick

Peter Paluch Thu, 01/26/2012 - 23:52

Hello Nick,

Accept my congratulations - and no, I do not smoke but thank you

Riccardo was faster, and of course, he's impeccably correct. Allow me to just add here and there a few comments.

Normally when customers go to providers for MPLS services, when is the type of service to be used talked about?

Throughout the negotiations, I would like to believe. You come to the SP and tell him what you need to accomplish, and he should suggest an appropriate technical solution and explain what that is - whether it is MPLS L3VPN, L2 pseudowires (AToM), VPLS, etc. Even the providers hides the technical details behind fancy (and unintelligible) names like "DS1 private port", there can be many MPLS services stuck behind that so definitely, you need to ask the SP what that exactly is.

If it is for the CE router.  Then I’m lost.  How does that work?

It is true that in MPLS VPNs, you are free to use your own addressing. However, the PE-CE link is somewhat special as both CE and PE must be on the same subnet, just as Riccardo suggested, and this involves an appropriate configuration of the PE router as well. Therefore, SP is telling you what IP address you are supposed to use on your CE WAN interface so that his PE configuration is well aligned. You are free to choose your own addressing in your internal networks behind the CE.

A link for how this configuration would/should look like would be great.

I am sure the MPLS books you've read about the MPLS L3 VPN show this nicely. But the simple rule is: the customer ignores the entire MPLS whatsoever. The configuration of the CE is totally agnostic of any MPLS specialties - you configure it simply with the IP addresses your provider asked you, assign your own internal addresses and run the routing protocol you have negotiated with your SP, and that's it. It is indeed as if you had the entire network for you alone and simply added a new router to the topology.

In MPLS VPN, is the configuration of CE routers completely independent of the provider’s knowledge?

Not entirely. You must agree on the PE/CE link addressings, and you must agree on the PE/CE routing protocol. But when that is done, nothing more is required at the CEs to consider the configuration correct and working.

If we wanted to used QoS and/or/with MPLS Traffic Engineering?  Does the  provider and the customer have to work together to share some  information to make this work?

Not in the configuration sense. But you need to work with your SP when explaining what your needs are, as Riccardo suggested. That may result in agreements about the committed data transfer rates, allowed priority markings (DSCP), recovery times, etc.

My two cents...

Best regards,

Peter

rsimoni Fri, 01/27/2012 - 01:27

"Peter & Riccardo Consulting" - it does not sound bad

Peter Paluch Fri, 01/27/2012 - 02:23

Riccardo,

"Peter & Riccardo Consulting" - it does not sound bad 

You bet!

Best regards,

Peter

nickhesson Mon, 01/30/2012 - 19:13
A link for how this configuration would/should look like would be great.

I am sure the MPLS books you've read about the MPLS L3 VPN show this nicely. But the simple rule is: the customer ignores the entire MPLS whatsoever. The configuration of the CE is totally agnostic of any MPLS specialties - you configure it simply with the IP addresses your provider asked you, assign your own internal addresses and run the routing protocol you have negotiated with your SP, and that's it. It is indeed as if you had the entire network for you alone and simply added a new router to the topology.

In MPLS VPN, is the configuration of CE routers completely independent of the provider’s knowledge?

Not entirely. You must agree on the PE/CE link addressings, and you must agree on the PE/CE routing protocol. But when that is done, nothing more is required at the CEs to consider the configuration correct and working.

So in reality, the customers do not control their own routing independently from the ISP anymore, when using MPLS VPN?  Because the PE routers need to participate in the routing, and customers of course don't have access to PE routers.  But I guess along with that, the configuration of the CE router's be comes very very simple.

  1. Configure the IP addresses on the interfaces.  One IP is told to you by the ISP, the other the customer picks.
  2. Configure a routing protocol to advertise the locally connected network(s) to the PE.
  3. Oh ya, set a password. 

But that it?  I'm I missing anything?

Thanks for everything guys,

Nick

rsimoni Tue, 01/31/2012 - 13:12

Hi Nick,

I am super busy for the rest of the week. If my 'associate' Peter does not step in I will answer this weekend.

Riccardo

Peter Paluch Tue, 01/31/2012 - 15:11

Hi Nick,

So in reality, the customers do not control their own routing independently from the ISP anymore, when using MPLS VPN?

This is not the way I would put it. Customers are still absolutely free to choose whatever addressing they are going to use on their individual sites. But a certain degree of coordination with the ISP is necessary because the CE and PE routers have to communicate - in plain IP routing (think ARP and the stuff) and IP routing protocols. You can't just invent some IP address on your CE's WAN interface and hope that the PE will be happy with that. Even though the PE placed the interface towards your CE into a separate VRF, there is still IP communication going between your CE and PE on the common link, and it has certain rules that must be upheld - regardless of VRFs or whatever.

So as I said, the ISP and the customer coordinate on the PE/CE addresses and on the common routing protocol, and that's it. Everything other is free for you to choose.

  1. Configure the IP addresses on the interfaces.  One IP is told to you by the ISP, the other the customer picks.
  2. Configure a routing protocol to advertise the locally connected network(s) to the PE.
  3. Oh ya, set a password. 

But that it?  I'm I missing anything?

That's it! You're not missing anything.

Best regards,

Peter

nickhesson Fri, 02/03/2012 - 00:30

So i was game or my head was clear up until today, where the ISP is telling me that we have to use BGP or I have to run GRE tunnels in order to run OSPF.    After reading everything, getting a feel.  Its seemed as if configuration is a common subnet, and routing protocols bewteen the CE to PE to distribute my local networks.  That would have been easy enough.  It looks like I have to use BGP. 

So the ISP has more control then everyone seems to think!  Meaning that there are dictating the routing protocol as well.

Does anyone manage their own CE routers connected to an MPLS network not using BGP? I would love to know, reason, it just seems as if, it would be just as easy to have the PE redistribute my OSPF routes in their vrfs using MP-BGP.

Nick

katerina.dardoufa Fri, 02/03/2012 - 02:25

Hello Nick,

even though the experts are more suitable to reply to your questions, I would like to take a turn, since I have been on the customer side just as you have.

My company is currently running GETVPN and the routing protocol used between our CE and the providers CPE is OSPF!

It was really hard to make them agree on this, since it does and has caused a lot of problems. As I understand, many ISPs use OSPF in their internal environment, so it does seem natural that they don't want to "blend it" with a customers OSPF.

Bellow I would like to mention a few of the problems we faced, by running OSPF:

  1. huge increase in OSPF database
  2. lost connectivity between our hubs and the PEs, due to excess LSAs and max-lsa configured for protection by the ISP.
  3. Lost connectivity when the provider tried to resolve issue 2, by not advertising some routes.
  4. We also wanted to use ISDN as a backup (running OSPF on our BRI), which resulted in routing loops and also lost connectivity as mentioned in 2
  5. Learning some routes from our spokes as E2 and others as O IA, because of different OSPF configurations on the providers PE, something which makes troubleshooting a bit odd.

Now we do have a pretty stable environment. The ISP has stopped advertising some routes, so the OSPF database is a bit smaller and we also use EIGRP for our ISDN connections.

In my opinion it is better and safer to start with BGP (I am sure that the ISP will help with the initial configuration) than try to fix problems as they arise!

Good luck with your implementation

Actions

Login or Register to take actions

This Discussion

Posted January 26, 2012 at 8:10 PM
Stats:
Replies:12 Avg. Rating:4.5
Views:2076 Votes:1
Shares:0
Tags: mpls, new_mpls
+

Related Content

Discussions Leaderboard