VPN ISP is different then internet navigation for users

[Mario Lessard]

Hi,

    My problem is a routing problem. I would like to configure a route for VPN connections with spoker router Cisco and also I would like to create a default gateway for the ISP VPN and an other default gateway for internet provider because it's not the same. 

I'm thinking to use BGP with next-hop but I think I did all combinaison and it's not working like I want.

Presently I have to create  a static route for each spoker like 

ip route 33.33.33.33 255.255.255.255 [default gateway ISP VPN]

and the default gateway for user internet is

ip route 0.0.0.0 0.0.0.0 [internet ip provider]

The problem with this setup, each time a spoker change ip address I have to find which ip now this router use and I add this ip route. It's not funny. 

What I try with BGP is something like that:

route-map GATEWAY permit 10

match ip access-list 1

ip next-hop [ip internet provider]

the access-list is base on source ip range like

access-list 1 permit 10.0.0.0 0.0.0.255

etc...

and apply it on closer port on the router. 

the default route is

ip route 0.0.0.0 0.0.0.0 [default gateway ISP VPN]

it seems to work but the problem is this setup seems to bypass the EIGRP route table. So, all source with ip range in access-list use next-hop. It's a problem for routing intranet. 

Brief, somebody as a solution for me? 

Two default gateway, one for ISP VPN and an other for ISP internet navigation for users.

Thank you

[Joseph Nelson]

Hi Mario,

Trying to understand your problem clearly. Can you provide a basic physical/logical topology diagram?

What do you mean by "ISP VPN" are you talking about a L3VPN WAN service? Or so you have a L2VPN service ( like VPLS, ENS etc)? If you are running EIGRP I'm assuming you have some sort of L2 WAN service. I'm also assuming you have multiple sites and you have a local internet connection?

Let me know...

[Mario Lessard]

I attach a quick schemas.

So, for VPN routing, I don't want to create a static route for each spoker and I would like that users continu to navigate through an other router like on the schemas. 

Presently default gateway is the router for internet users. I want to create a tunnel DMVPN more dynamic and replace all static route for only one like

ip route 0.0.0.0 0.0.0.0 (internet provider for VPN)

because presently the ip route is

ip route 0.0.0.0 0.0.0.0 (router internet user gateway)

so I can't have to gateway accept I think with route-map

Thank you

[Joseph Nelson]

Hi,

Can you post your config for a "spoke" and for the "VPN router"? After banging my head against the wall for 30 minutes, I think I understand what you want to do. Your original reply was from the perspective of the "VPN router" in your schema; below I summarize your situation and your goal:

Right now, your DVMPN traffic and your "general internet browsing" traffic is traversing the "Internet User Gateway" device in your schema. You would like to force your DMVPN to use your "Gateway for VPN Internet" device in your schema. Currently, when your spoke changes IP address, your DMVPN breaks because of your static route you put into the "VPN router" to force traffic to go via the "Gateway for VPN Internet."

I assume that the only devices you control are the "spokes" and the "VPN router". All other devices are outside your control. I assume also that your "VPN router" is the hub in your DMVPN network.

You mentioned you use BGP a few questions may lead to a solution:

• Do you do BGP with both your "VPN ISP" and your "Internet ISP"? Do you get full routes from both ISPs?
• Do all of your spoke sites have IP addresses that come from the same ASN ( lets call this ASX)

I want to make sure I understand your configuration, problem, and setup before I can offer the solution I'm thinking of.

Let me know...

[Mario Lessard]

Hi Joseph,

      The summarize is exactely the situation. For the configuration I'm not very confortable with that. This forum is public and a lot of person could see it. A lot of information could provide with a configuration file. Maybe we could have a private communication? 

      For the question, the BGP it was a test with no result. I did some combinaison but maybe not the right one:

access-list 1 deny 10.0.0.0 0.255.255.255
access-list 1 deny 172.16.0.0 0.16.255.255
access-list 1 deny 192.168.0.0 0.0.255.255
access-list 1 permit any

interface GigabitEthernet0/1
 no ip policy route-map GATEWAY_TO_CAS

route-map GATEWAY_TO_CAS permit 10
 match ip address 1
 set ip next-hop [internet provider for VPN]

or

route-map GATEWAY_TO_CAS deny 10
 match ip address 1
route-map GATEWAY_TO_CAS permit 20
 set ip next-hop [internet provider for VPN]

but with access-list permit....

Let me know for spoke and hub configuration, I could provide the config more private.

Thank you

[Joseph Nelson]

Hi,

I completely understand the need for privacy. You can private message me j0nelson<at>yahoo<dot>com

The solution I'm thinking of involves using BGP with your two providers. If all of the following are true:

• You have an eBGP session with both providers at your hub site. Specifically, "VPN Router" in your schema has an eBGP session "VPN ISP " device and "Internet User Gateway" device
• All of your spoke sites are in the same ASN ( or are in a small group of ASNs you can match with a regex

Then

• Only accept prefixes originated from "VPN ISP" from the "VPN ISP" peer
• Only accept a default route from the "Internet User Gateway" ISP

This would be a solution because the DMVPN tunnel destinations are only known via the "VPN ISP" peer. Everything else follows a default router to the "Internet User Gateway."

[Joseph Nelson]

Mario,

Firstly, thanks for putting your trust in me-- I know its huge to give a complete stranger your configuration, let alone some guy on the support forums with a non-existent reputation :)

Based on your configuration, even the solution I provided using eBGP isn't applicable because:

• Hub doesn't run BGP at all
• Spokes aren't on the same ASN ( I see you have a Phase2 DMVPN config and some static tunnels)

Your challenge is interesting but I don't think traditional solutions will work. You want to Traffic Engineer the DMVPN tunnel destination traffic, however, that requires a priori knowledge of those tunnel destinations. My solution attempted to provide that knowledge implicitly with BGP-- this would work if all your spoker sites used the same ISP ( I see there are groups of IPs in the same blocks, however) as all IPs would be in the same ASN. When you look at things like policy-based routing ( which is the go-to standard for this type of TE), the limitations are severe: You can only match on "ip length" and "ip address". The former requires a priori knowledge of the tunnel destinations as mentioned above. The latter doesn't really apply;  I thought about matching based on the GRE-encapsulated packet length but you don't know this length until _after_ encapsulation.

Now, lets turn to some other mechanisms:

• Are there are set of host behind the hub router ( say a set of servers) that your spoker sites would always hit? These could be terminal services boxes that spoker site users need to login to. If so, you could use policy-routing to match those servers as the source and policy-route out the correct interface
• Consider, why you need to segment DMVPN traffic and "Internet Browsing" traffic? Is it because you ISP VPN is a faster link? What if you enabled QoS on the hub to guarantee bandwidth to the DMVPN overlay and then scavenged the regular "Internet Browsing" traffic. This is easily done. This way DMVPN and Internet Browsing traffic use the same egress point to the Internet with guarantees on DMVPN performance and a failover path to the other ISP

Hope this helps you some. Let me know your feedback.

[Richard Burts]

Perhaps there are things in this discussion that I do not understand or recognize their implications. But it seems to me that DMVPN would provide pretty much what the original poster seems to want. With DMVPN you should be able to have a default route point at one interface and to establish the multipoint tunnel of the hub on a different interface. With DMVPN there is no need to configure static routes for the spoke sites since the spoke sites dynamically find the hub and register with the hub. And if you run a dynamic routing protocol over the DMVPN then the hub routing table is automatically populated with routes to the spoke networks with the DMVPN tunnel as the next hop.

With DMVPN there is no need for BGP for the spokes or for Policy Based Routing for the spokes.

HTH

Rick

[Mario Lessard]

Yes but I want also a redundancy with 2 hub. It's what I have now.

[Richard Burts]

There are implementations of DMVPN with two hubs. So that should not be a problem.

HTH

Rick

[Mario Lessard]

But I have to add this commun device (router) that I don't have anyway presently? 

Presently it's working but it's not perfect. I have to add in the hub a static route for the spoker because the default gateway it's not provider of Hub but it's an other router that I don't manage to give access to user in LAN part internet. 

Because Internet for user in LAN is not the same then the internet for DMVPN for Hub and spoker. 

So, my goal is remove all static route for spoker and have only one. But if I understand I can't do it with only the hub, I have to add an other router that the function will be the routing with internet for user and routing to hub for remote VPN communication to the spokers (EIGRP).

That I'm understand. Because I can't use route map on the hub to try use a different route for user then for VPN. 

Thank you

[Joseph Nelson]

Mario,

Here's what I would do. See attached diagram ( made in www.draw.io, can send you the xml b/c I can't upload it on this site). In this design, the path from the Hub site is driven by the DMVPN IGP so you don't have to worry about the tunnel destinations at all.

This whole time I was assuming that your "VPN Router" was the only thing between the "Users" in your diagram. Is there anything downstream from that device?

Can you post a _complete_ diagram?

[Joseph Nelson]

I'd like to discuss some specific features of the solution I proposed.

This one is an interesting one, partly because I've been guessing at the topology and partly because there a subtle problem the OP is running into when the DMVPN and Hub Site routing are collapsed ( as I have strong evidence to believe). Again, a complete topology will help verify.

If the OPs DMVPN hubs are the also routing for the hub site AND spokes do not use their DHCP learned default, the OPs problem occurs. Consider the traffic flow for a visit from a spoke LAN computer to www.google.com:

• Traffic hits spoke router, follows default route learned via EIGRP, sends toward hub1 via DMVPN tunnel
• At hub1, traffic comes out of mGRE tunnel, finds it needs to use default route to get to www.google.com, consider what happens depending on the default route that is used:
  • Hairpining: If hub1 uses ISP1 learned default, traffic goes back out ISP1
  • Asymmetric DMVPN: If hub1 uses ISP2 learned default, traffic goes to ISP2--that also means the DMVPN traffic goes to ISP2. The tunnels from the hub1 take ISP2 to get back to the spokes

And this is the subtle crux of the problem. If hub1 uses ISP1, then the ISP1 path can be saturated in either direction. Suppose spoke user does http traffic-- page request from spokes are small, downloads may not be. Suppose youtube video, ISP1 inbound will be swapped, so will outbound leading to possibly disruption in DMVPN. One spoke user degrade all spoke sites by watching an HD video.

But the problem is caused _because_ DMVPN and Hub site routing are collapsed. Letting the spokes split-tunnel internet bound traffic solves the hairpinning problem. This still leaves the Asymmetric DMVPN problem-- to solve this, you either tell hub1 to default to ISP ( leading back to hairpinning), you tell hub1 to route specific tunnel destinations out ISP ( leading to original problem in the post), or you let hub1 use ISP2 path ( leading back to Asymmetric routing problem). These two problems exists without even beginning to deal with the problem of ensuring hub site users always use ISP2 path.

My solution of separating the hub site routing solves both problems at the cost of introducing another router. In the traffic flow scenario, traffic to www.google.com never makes it to the hub1 router--instead it egresses at the spokes using their default route ( no hairpinning). Next, hub1 is free to use whatever default it wants--to meet the goal, we'll put a static default route pointing to ISP1 and an EIGRP-learned route learned from the core ( or from hub2 directly). The tunnels will use the ISP1 path and failover to ISP2 if needed ( no Asymmetric DMPVN problem). Lastly, the goal that hub site users use ISP2 for internet browsing is accomplished simply: ensure core box has a default pointing to ISP2.

[Joseph Nelson]

Richard,

The OP already has a DMVPN ( Phase2) setup. As outlined in the thread, the following are true:

1. Two internet connections at the hub site; One connection used for DMVPN (ISP1), One used for general internet browsing (ISP2)
2. OP wants to force traffic to DMVPN spokes out of ISP1
3. OP does not want to use static routes at hub to achieve Goal #2

So the problem is really how to Traffic Engineer toward the DMVPN spoke sites-- in effect, the question is, how to TE DMVPN tunnel destinations. This of course requires that you know the tunnel destinations before hand ( a priori knowledge)-- because the spoke sites don't have static IP addresses, you will never have this a priori knowledge of the tunnel destination-- ergo the OPs problem.

The eBGP solution was given as an attempt to implicitly provide knowledge about the spokes and draw traffic toward ISP1. Indeed if OP hub device has eBGP connection with ISP1 and all spoke sites come from same ASN, then eBGP can be used to select the path using ISP1. The assumption that all spokes sites are from a common ASN ( or a small set of ASNs) is a fair one, since the business typically try to stay with a single ISP.

It was never specified how whether or not TE was desired at the spoke side, however, I would assume that all non-enterprise traffic would drop out of the DMVPN at the spoke ( i.e. split-tunnel). 

This is an interesting problem, but I don't see how it can be solved purely in routing:

• Either you know where your DMVPN spokes are coming from; Use routing protocol to select the same path back ( i.e. eBGP based) OR
• You known where your DMVPN spoke users are going to on the hub LAN; Identify the return traffic and policy route it via ISP1 OR
•  Your put a static route to force spoke-bound traffic over to ISP1; And wait for the site to go down because the spokes IP address changes ( as is the case)
• Separate the DMVPN and Internet routing to two separate devices; A core device could have static routes for spoke subnets the next-hop being this new DMVPN router. The core device also has a floating static for Internet routing. I try not to rearchitect the OP's network so I didn't suggest this earlier. 

As I hinted at, a possible solution would be just letting general internet and DMVPN traffic share the same link. 

Joe

[Joseph Nelson]

Richard,

See my latter post for a detailed analysis of the OPs situation. But I wanted to address your comment:

"With DMVPN you should be able to have a default route point at one interface and to establish the multipoint tunnel of the hub on a different interface."

It depends on your perspective:

• Your a spoke, hub is on the internet. You configure at most two NHS. You could add a static route for the hubs and use a different interface as you imply
• Your a hub, spokes are on the internet. You can't added a static to the spokes because you don't know their IP until after mGRE and NHRP step in. When/if the spoke IP changes, site goes down. In a situation where the spoke IP changes, the interface used for default routing will be the path the DMVPN tunnel takes toward the spoke.

As outlined in my other post, there are operational challenges to default routing at the hub.