Routing internet traffic over MPLS circuit?

[david]

I've been reading for hours, but nothing seems to hit the nail on the head.  This has to be a common scenario? We have multiple sites connected together via managed MPLS.  Each site has their own local Internet connection / ASA firewall which is accessed via static default route.  MPLS is only used for site to site WAN traffic.  MPLS routing is handled by OSPF.  Every once in a while the Internet connection at a remote site drops so I'd like the ability to reroute that Internet traffic out our main site Internet connection if a remote site ISP goes down.  I'm aware of the default-information originate command, but I only want to route Internet traffic over MPLS if the local Internet is unavailable.  Is there a way to do this using tracking/metrics?  Thanks!   

[Robert Falconer]

What is the layer 3 device at each site? You'll need something that can peform an IP SLA.

I would think the easiest way to do this is to set up an IP SLA with an echo to someplace just past the upstream connection of the ASA. Maybe 2 or 3 hops away. Then set a static host route for that IP pointing to the ASA so it can't get to the echo destination through the MPLS link. Or you can deny this traffic on your egress firewall at the hub. It's nice to be able to control where this traffic goes.

Then set the default static to the ASA to use the SLA for tracking. If it goes down, it will drop out of the routing table and your OSPF learned default should install. This should also work if the ASA fails.

I do something like this now with my MPLS connections.

ip sla 1
icmp-echo x.x.x.x
frequency 10
exit

ip sla schedule 1 start-time now life forever

track 10 ip sla 1 reachability
delay down 10 up 10

ip route x.x.x.x 255.255.255.255 ASA
ip route 0.0.0.0 0.0.0.0 ASA track 10

[david]

Thanks, yes, we have Cisco 3750G's at each site. I just whipped up a diagram which should help explain our setup (see attached).

Mfurnival, that's probably the best bet since I'm also trying to guard against the opposite problem, whereas the MPLS circuit at a site goes down.  In that situation, I want the path between offices to use an IPSec L2L tunnel between ASA's.  I have the tunnels created and have been using a floating static route on the 3750 with higher cost than the OSPF route, but it's become too difficult to manage.  My goal is to automate the whole process.       

[Jon Marshall]

It all depends on whether you receive a default route from your ISP at each site  or not.  This would mean you would be running a dynamic routing protocol between each ASA and their corresponding ISP which i suspect you aren't.

If the ASA has a statically configured default route then tracking is really your only option.

You can guard against both failures using IP SLA on the 3750 as suggested by rmf.newport.

You don't need floating statics for any remote site subnets because if any specific remote site subnets are lost, for whatever reason, your static default route (which you are tracking) would send the traffic to the ASA.

So you simply need -

1) a default route generated from the main site for backup internet. It would have a higher AD than the locally defined static route so would not be used unless your trracking fails.

2) at each site a static route pointing to the local ASA with tracking which covers both local internet and any sites that lose connectivity via MPLS.

Jon

[david]

Thanks Guys, great information!

Jon, ASA's have statically assigned default routes.

So to take this one step at a time and verify the plumbing, could I >

• Add "default-information originate" to my OSPF process at our main site
• Check the routing table at remote site to verify it knows of both static and OSPF default routes (sh ip route, sh ip route ospf).
• Temporarily remove the static default route at remote site and verify that the OSPF default route becomes active and that I can access the Internet from the remote site via the main site.

[Jon Marshall]

You could do what you suggest to test it.

As long as you have both default routes on the 3750 but under normal conditions the one pointing to the ASA is in the IP routing table it should work fine.

Jon

[david]

This is why I like to do things in steps... :)  I should probably start another thread for this issue, but for some reason my default-information originate is not propagating.  On the router I applied it to I see it in the database (sh ip ospf data) as a Type-5 AS, but it never shows up in my OSPF database at remotes sites?  I labbed it up via packet tracer and it works fine there so I'm not sure what's different in my prod network.  Any thoughts to further troubleshoot?  Do I possibly need to do a "clear ip ospf process" after adding the "default-information originate" command to an already established OSPF process?  I didn't try that yet since it will cause a brief outage.

Main router pertinent info >

router ospf 1
 network 10.10.0.0 0.0.0.255 area 10
 network 10.20.0.0 0.0.0.3 area 0
 network 10.30.0.0 0.0.0.255 area 10
 default-information originate

ip route 0.0.0.0 0.0.0.0 10.30.0.2

sh ip ospf data >

 Type-5 AS External Link States

Link ID         ADV Router      Age         Seq#       Checksum Tag
0.0.0.0         10.30.0.1       1239        0x80000019 0x00AD99 1

[Jon Marshall]

On a remote site when you do a "sh ip route" what do the main site subnets show up as ie. what type of OSPF route ?

Jon

[david]

O E2     10.10.0.0/24 [110/20] via 10.20.1.2, 19:44:20, Vlan20

I also opened a ticket this mornig with our provider to see if they're doing something within their network that may be blocking LSA Type 5.  Still waiting for their response...

[Jon Marshall · Accepted solution]

Because it is showing as an E2 route it looks at first glance as though you are peering between the CE and PE with BGP not OSPF.

In your diagram there are MPLS managed routers. Do you know if these are the CE devices ?

If so and if they are managed by the provider they may simply be redistributing your OSPF routes into BGP on the CE device in which case you are not actually passing OSPF routes across the MPLS network.

See what your provider says but if the CE is peering with PE using BGP another option may simply be to generate the default route within BGP on the main site CE device.

Let us know what the provider says.

Jon

[david]

Jon, you were right!  they are using BGP on their backbone and were not allowing my default-information originate command to propagate.  They resolved it on their end and I now see my ospf default route so now I can test it over the weekend.  Thanks!  btw - *tried assigning correct answer, but it kept complaining that the link was broken so I'll try again over the weekend.  Cheers!

[Jon Marshall]

No problem.

Let us know if you have any more issues with the testing.

Jon

[mfurnival]

Do you really need IP SLA here? Why not just run some sort of dynamic routing protocol between each site ASA and the router? If the ASA loses the ISP route then it stops advertising it and you have a backup static route pointing into the MPLS cloud with a lower admin distance / metric.