Multiple Default Routes into MPLS cloud

ftikphillips · ‎12-07-2009

I have 2 data centers each with Internet connections, one on the east coast and one on the west coast connected to a MPLS network. I also have a bunch of remote sites around the country connected to the same MPLS network.

I can advertise the default route into the MPLS network from both sites and sites closest to the west coast data center will prefer that default route and sites closest to the east coast data center will prefer the default route to the east coast data center.

Since the east coast data center's Internet bandwidth is used more than the west coast data center's Internet bandwidth. There are times that I would like to some how tweak the east coasts advertisement of that default route so that the larger east coast offices will route out the west coast data center during periods of high utilization on the east coast.

I peer with my provider via BGP and the provider runs OSPF. When i tweak attributes in the BGP advertisement of the default route from the east coast it is an all or nothing update in that all remote sites then go out the west coast data center.

Is there anyway of solving this problem?

Thanks in advance.

Giuseppe Larosa · ‎12-08-2009

Hello Ftikphillis,

considering the following notes:

>> I can advertise the default route into the MPLS network from both sites and sites closest to the west coast data center will prefer that default route and sites closest to the east coast data center will prefer the default route to the east coast data center.

and:

>> I peer with my provider via BGP and the provider runs OSPF. When i tweak attributes in the BGP advertisement of the default route from the east coast it is an all or nothing update in that all remote sites then go out the west coast data center.

It is your service provider that helped you to implement this routing, because it is a L3 MPLS VPN.

When you say I notice that some remote sites prefer to use east datacenter to go to the internet the choice is made on the service provider PE node connected to your remote site not on the remote site itself.

You need to work together with service provider:

by using eBGP for PE-CE session you have faculty to set and send BGP communities, you can use a BGP community to flag a route or a set of routes.

In this case what is interesting is that the two default routes can use two different BGP communities.

Service provider can agree on accepting the routes and to propagate the BGP community attribute with the route.

At each PE node serving a remote site a change in configuration can make it prefers the default route with a certain BGP community by setting an higher BGP local preference even if the PE node is nearer to the other exit point in terms of SP IGP metric to BGP next-hop.

at datacenter:

ip prefix-list only-default permit 0.0.0.0/0

route-map setBGPcom permit 10

match ip address prefix only-default

set bgp community 5000:100

route-map setBGPcom permit 20

! other criteria here for other specific prefixes

router bgp 5000

neigh isp.pe.address route-map setBGPcom out

! next required to actually send BGP community attribute on the eBGP session

neigh isp.pe.address send-community

Service provider has to implement appropriate changes to accept and propagate BGP communities

on PE node serving a remote site

ip community-list 11 permit 5000:100

route-map prefer_selected_routes permit 10

match bgp community 11

set local preference 200

route-map prefer_selected_routes permit 20

! to accept other vpnv4 routes

the route-map has to be applied on the vpnv4 address families on the BGP sessions with route reflector servers

router bgp ISP_ASN

address-family vpnv4

neigh RRS1 route-map prefer_selected_routes in

neigh RRS2 route-map prefer_selected_routes in

!

As you can see this is possible but not straightforward, without service provider cooperation you cannot achieve the desired result

Hope to help

Giuseppe

vdadlaney · ‎12-08-2009

ftikphillips wrote:

I have 2 data centers each with Internet connections, one on the east coast and one on the west coast connected to a MPLS network. I also have a bunch of remote sites around the country connected to the same MPLS network.

I can advertise the default route into the MPLS network from both sites and sites closest to the west coast data center will prefer that default route and sites closest to the east coast data center will prefer the default route to the east coast data center

Hi,could you please elaborate on how you are achieving this since in your post you mention you are running BGP. By default BGP will only prefer one path as the best path unless your provider is using maximum paths which even than per my understanding will not provide the connectivity you are referring to as both paths will be installed and traffic will be load shared across both paths. Am I misunderstanding something over here? I am assuming that the provider has a PE device/s in both regions and the provider cloud is a single AS and there is no AS path manipulation on the provider side when they advertise the default route between their PE's across the MPLS cloud. Really appreciate if you could confirm this.

Since the east coast data center's Internet bandwidth is used more than the west coast data center's Internet bandwidth. There are times that I would like to some how tweak the east coasts advertisement of that default route so that the larger east coast offices will route out the west coast data center during periods of high utilization on the east coast.

As Giuseppe has mentioned already the provider would have to get involved in order to provide this dynamic functionality as I cannot think of a solution where you would be able to achieve this on your side without the providers input (But ofcourse I am by no means an expert and would really be interested in learning should you find another solution). I would think that the provider would have to set up some sort of Traffic engineering based solution based on load to achieve this sort of functionality. I would appreciate if you could share the solution once you have spoken to your provider. thx

tk95439 · ‎01-01-2011

Maybe a little past being useful as the last post was over a year ago...

But if you are using an MPLS VPN you could try having the provider set you up with 3 VIPs. Advertise your wan routes in (just an example route target) 65000:1. A default from the west cost into 65000:2 with no as path prepend and a default with a prepend from the east cost. Then the opposite in 65000:3 where east sends a default with no prepend and west prepends. Than for each site you could accept routes from 65000:1 and either 65000:2 or 3. Giving you an option per site which data center to try first.