PBR for IKE + ESP in Dual-Provider DMVPN

[mbrown]

I'm having some trouble with dynamic spoke-to-spoke tunnels in my DMVPN topology. Essentially, each spoke uses a T1 at Serial0/0/0 for the primary DMVPN network and a T1 at Serial0/1/0 for the primary Internet circuit. Static routes with varying administrative distance values send traffic to the desired interface while allowing both Internet and DMVPN traffic to share an interface if one T1 drops.

Everything works well except for spoke-to-spoke tunnel creation. Basically, spoke-to-spoke IKE traffic that I want to send out the primary DMVPN T1 is following the static default route out the other T1 designated for Internet traffic.

I've tried using PBR to send this traffic out the correct interface, but it doesn't seem to have any effect. I've configured an access-list matching the source IP of the IKE traffic, a route-map matching that ACL and setting the next-hop interface (serial w/ ppp encapsulation), and enabled that route-map with "ip local policy route-map mytestmap".

Does local policy based routing have any effect on IKE and/or ESP?

Thanks in advance for your help!

-Mason

[aghaznavi]

Try these links:

http://www.cisco.com/en/US/products/ps6350/products_configuration_guide_chapter09186a0080455c71.html#wp1039515

http://www.cisco.com/application/pdf/en/us/guest/products/ps6658/c1161/cdccont_0900aecd80313ca0.pdf

also check this bug-id:CSCed68627

[mbrown]

Thanks for the links and the info. Unfortunately our setup falls just beyond any of the standard scenarios described, and the behavior is understandable, but the fix I'm trying to implement (policy-based routing) isn't behaving as I expect.

The problem arises because we have a static default route out the second carrier's T1. Spoke-to-spoke IKE and ESP traffic that should go out the first carrier's T1 are instead sent out the second carrier's T1 (following the default route). Each T1 has its own carrier-provided IP block and only use a static default route on the secondary interface (used for Internet traffic) and a floating static default route on the primary (as a backup for Internet traffic).

If I add static routes for spoke-to-spoke traffic to use the primary T1, things work 100%. But that ruins the dynamic goodness that comes with DMVPN.

So I'm trying to use policy-based routing to send the spoke-to-spoke traffic across the primary interface (instead of following the static default route out the second interface). But it doesn't appear to be working.

So I'm wondering if there's anything about IKE, ESP, or GRE traffic that can't be policy routed. Maybe something related to hardware acceleration?

-Mason

[mbrown]

Here's a simpler take on what I want that isn't working....

Spoke 1 Serial 0: 3.3.3.3

Spoke 1 Serial 1: 4.4.4.4 (default route)

Spoke 2 Serial 0: 7.7.7.7

Spoke 2 Serial 1: 8.8.8.8 (default route)

It works if I configure:

(Spoke 1)

ip route 7.7.7.7 255.255.255.255 serial 0

(Spoke 2)

ip route 3.3.3.3 255.255.255.255 serial 0

It doesn't work if instead of static routes I do:

(Spoke 1)

access-list 101 permit ip host 3.3.3.3 any

route-map test permit 10

match ip address 101

set int s0

ip local policy route-map test

(Spoke 2)

access-list 101 permit ip host 7.7.7.7 any

route-map test permit 10

match ip address 101

set int s0

ip local policy route-map test

With the policy routing configured above, I can see ICMP and TCP packets being policy routed correctly. I even see IKE packets policy routed correctly. But something after that isn't policy routed properly. I don't know if it's ESP, NHRP, or what, but it doesn't work like it should, and the NHRP resolution on the destination spoke is never completed. The static route example works perfectly.

-Mason

[mbrown]

Well I finally found the right search on Google to get the answer to my question: +PBR +NHRP

I came across this link:

http://puck.nether.net/pipermail/cisco-nsp/2006-May/031088.html

which led me to these Cisco Bugs:

CSCef95982

CSCef66719

Both bugs describe the issue I'm seeing, and the workaround section states that this is expected behavior. Apparently GRE isn't supposed to be policy routed, and the workaround is to put static routes for remote tunnel endpoints with the desired next hop / outbound interface.

This is really annoying though, because everything else in DMVPN provides such a cohesive dynamic environment where adding a new spoke requires ZERO configuration on the hubs or other spokes. With this restriction, each additional spoke will require reconfiguration on every spoke (we have about 100 spokes so far).

If anyone from the DMVPN development team reads this, please consider fixing the bugs listed above to allow policy-routing of GRE traffic to prevent the static route requirement that comes with dual-ISP connectivity in full-mesh DMVPN.

Thanks!

-Mason