Hi Federico and thanks for reply. Yes in deed, this one is one of the best documents on DMVPN and actually I have been working with it for a while now. The problem here is, the whole setup works fine (I recreated it in my lab) but as long as the 2 hubs are on the same subnet (the WAN side). When I put them on 2 different subnets (as in case of having 2 different ISPs connected to each hub) it does not work es expected.

I haven't gone to the NAT stage yet because of that. Supposedly the dual hub with single cloud/layout and having the 2 hubs on different subnets work, how do I connect the 2 hubs to the 2 different ISPs? When you configure the secondary hub, it points to the primary hub (as it were spoke itself) but here what IP address should I use to point to the primary hub from the secondary? Do I use post-NATed IP address of the primary hub (here the public IP of the primary ISP) or I should use the pre-NATed IP address of the primary hub (private IP address)? With the second option I would need to have both the ISP routers and DMVPN hub routers connected together through a switch and on the same subnet I guess right?

Or maybe I should go for dual-hub dual cloud/layout so each hub would represent a separate vpn cloud?

The SPOKEs will be running PAT and ZFW firewall on themselves so I guess there is no problem with that. I also read I should not have a problem with HUBs behind ISP routers (I guess the ISP boxes do PAT for user Internet access) as long as we do static NAT to the DMVPN HUB on the ISP box.

What's your opinion on that? Thanks.

Are you pointing the second hub to the public IP and the tunnel IP of the primary hub?

When you say that it does not work as expected when you do this, what exactly seems to be the problem?

Federico.

Correct! In my lab the public IP of the primary hub is 172.16.20.2/24 with the tunnel being 10.0.0.1, the secondary hub has public IP set to 172.16.10.2/24 and the tunnel 10.0.0.2. So to speak they are on two different public subnets and the secondary hub points to 172.16.20.2 and 10.0.0.1. The spokes routers should have 2 OSPF route entries to the private subnet behind the hub routers but they only have one. I have the 2 hubs connected together to the internal subnet 192.168.10.0/24 with primary hub being 192.168.10.1 and the secondary 192.168.10.2. So the spokes should learn the path to 192.168.10.0/24 through OSPF via 10.0.0.1 and 10.0.0.2, I don't get that on the spokes. However this works fine if I have the 2 hub's public IPs on the same subnet, say 172.16.20.2/24 and 172.16.20.3/24, exactly as in the Cisco document...

I can still ping the both private  IPs, 192.168.10.1 and 192.168.10.2 from the spokes but on all the spokes I have only one IA entry to the 192.168.10.0/24subnet through 10.0.0.1 tunnel. When I do ping from one of the spokes to the internal host on 192.168.10.0/24 and disconnect the primary hub, there is a failover to the the secondary hub after about 20-30sec but when the primary hub comes back on line, the host on 192.168.10.0/24 stops responding for good. I can no longer ping 10.0.0.1 or 10.0.0.2 from the spoke.

I have all set as per Cisco document but it does not give me expected results when I put the 2 hubs on 2 different subnets. That should work because it sounds logical. All I want to achieve is to have 2 hubs behind different ISPs routers doing NAT, obviously for redundancy purposes...

Thanks for further input on that.

Remi

So, when you have both hubs sharing the same WAN subnet, everything works...
It means, that on the Spoke router, you see the NRHP mappings for both Hubs, both IA OSPF entries and both crypto tunnels established
correct? Everything normal, let's see what the problem is in the second scenario...

When you switch to having both Hubs in separate WAN subnets, do you still see the NHRP mappings for both Hubs on the spoke?
Do you see on the Spoke both Hubs as OSPF neighbors?
Are you getting any log or error about receiving non-IPsec packets?

Federico.

OK, I did some more test today...thanks for the follow up by the way, I appreciate that. I get everything right as you asked, on both spokes I get NHRP mapping to both hubs - 10.0.0.1 and 10.0.0.2, so that to the internal network behind the 2 hubs I have 2 route entries in the routing table. I tried to manipulate the distance value under OSPF so that the route through the secondary hub would be least preferred. That somehow does not work so I still have 2 routes through both hubs in the routing table on the spokes.

What happens next, I take the primary hub offline, again the fallback into the secondary hub (in this case I am pinging from the spoke router to the internal host on the network behind the hubs) takes around 30sec, I check the routing table on the spoke and now we have only one route to the internal network behind the hubs but of course only through 10.0.0.2 - secondary hub. Now when I put the primary hub back online, there is no fallback into the 10.0.0.1 route, in fact the primary route never comes back up and the responses to the PING stop forever. I can't even ping the other end of the tunnels 10.0.0.1 and 10.0.0.2, it seems like to whole network goes down after the primary hub fails. What I can't quite get is the non-IPSec packets on the primary hub, exactly as you mentioned. Here is an example:

%CRYPTO-4-RECVD_PKT_NOT_IPSEC: Rec'd packet not an IPSEC packet. (ip) vrf/dest_addr= /172.16.20.2, src_addr= 172.16.40.2, prot= 47

here 172.16.20.2 is the public IP of the primary hub and 172.16.40.2 public IP on the spoke. What could be causing that? I have the crypto config same on all the routers.... I am not sure if that starts happening after the primary hub goes down or from the very beginning, I need to verify that but something tells me that happens from before. I am using the primary hub as 2801 with IOS 15.

Thanks for your help.

I am experiencing the same problem, has there been a resolution to this issue? 

I am going to set up a lab to try this.

Hopefully, I'll let you know soon.

Federico.

Cool, thanks a lot for that. I am also carrying on with my tests and will post some results soon. So far I set up one DMVPN cloud (no redundancy) with the hub being behind an IPS router that does NAT and PAT and with the spokes running DMVPN, PAT and CBAC on the same box. All works good (with EIGRP) but can't establish spoke-to-spoke tunnels. There is a connection between the spokes but through the hub, so the hub does the whole double encryption for the inter-spoke communication...

I need to rush for dual DMVPN cloud (redundancy model) so I am going to try dual hub layout with with a single cloud as well as dual cloud. The thing is how do I set up the secondary hub to speak to the primary one? Do I connect the 2 hubs independently to each other's ISP or I put the IPS routers and the hub on the same subnet? The 2 hubs will connect to the same LAN and from my previous test (no NAT) when I pointed the secondary hub to the primary hub over the global inside IP address on the secondary hub, the routing table on the secondary hub had an entry to the primary hub through the LAN, is that correct? This is how it shoud be?

Hi Federico,

Did you have a chance to set up your lab yet? I went ahead and did a dual hab with dual cloud setup. I also included NAT on the hub side (a router sitting before each hub and doing static NAT) and PAT on spokes. All seem to be working fine up to the point of simulation of the primary hub failure. After the failure there is a 20sec fall back into the secondary clound routed through the secondary hub. When the primary hub come back up on line, the primary route is never reestablished...Moreover the primary tunnel end 10.0.0.1 does not respond to PING from the spokes however it seems to be up.

I am not sure if it's nothing to do with the EIGRP routing. I set different delay values on both tunnels on the spokes, I increased very much delay on the secondary tunnel so that it would always be least preferred. I have "crypto isakmp keepalive 20" on both hubs and spokes.

Here I am posting results of the "sh ip eigrp 1 topology detail-links":

SPOKE 1:

dmvpn-spoke1#sh ip eigrp 1 topology detail-links
IP-EIGRP Topology Table for AS(1)/ID(192.168.30.1)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 10.0.0.0/24, 1 successors, FD is 2816000, serno 22, Stats m(52)M(52)A(52)c(1)
        via Connected, Tunnel0
        via 10.0.1.1 (3353600/3084800), Tunnel1
P 10.0.1.0/24, 1 successors, FD is 2828800, serno 23, Stats m(48)M(48)A(48)c(1)
        via Connected, Tunnel1
        via 10.0.1.1 (3097600/2828800), Tunnel1
P 192.168.40.0/24, 1 successors, FD is 3123200, serno 34
        via 10.0.1.1 (3123200/2854400), Tunnel1
P 192.168.10.0/24, 1 successors, FD is 2818560, serno 29
        via 10.0.1.1 (2831360/28160), Tunnel1
P 192.168.30.0/24, 1 successors, FD is 281600, serno 1
        via Connected, Ethernet1/0
        via 10.0.1.1 (3123200/2854400), Tunnel1
dmvpn-spoke1#

SPOKE 2:

dmvpn-spoke2#sh ip eigrp 1 topology detail-links
IP-EIGRP Topology Table for AS(1)/ID(192.168.40.1)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
       r - reply Status, s - sia Status

P 10.0.0.0/24, 1 successors, FD is 2816000, serno 13
        via Connected, Tunnel0
        via 10.0.1.1 (3596800/3084800), Tunnel1
P 10.0.1.0/24, 1 successors, FD is 3072000, serno 37
        via Connected, Tunnel1
        via 10.0.1.1 (3340800/2828800), Tunnel1
P 192.168.40.0/24, 1 successors, FD is 281600, serno 1
        via Connected, Ethernet0
        via 10.0.1.1 (3366400/2854400), Tunnel1
P 192.168.10.0/24, 1 successors, FD is 3074560, serno 38
        via 10.0.1.1 (3074560/28160), Tunnel1
P 192.168.30.0/24, 1 successors, FD is 3366400, serno 39
        via 10.0.1.1 (3366400/2854400), Tunnel1
dmvpn-spoke2#

Would be able to analyse those routes and tell if the 10.0.0.0/24 is still most preferable route?

Thanks in advance.

Both spokes have the 10.0.0.1 directly connected via tunnel0.

I havent' been able to make a lab (caught up in other things, I promise I'll try to do it shortly)...

So, do you have the dual hub on two separate subnets working now (with the exception of the failover?)

Federico.

Correct. I have dual hubs on two seperate subnets working fine. There is comunication between the spokes and between hubs and spokes on both subnets and of course between all the LANs behind spokes nad hubs.

The following still is not working:

- no failover, well there is failover into the secondary subnet but not fail back into the primary subnet, after bringing back online the primary hub, there is no communication between hubs and spokes,

- no direct communication between spokes, all the encrypted traffic goes through hub.

I look forward to your results.

Remi

The fact that all traffic is being sent through the hub between spokes has to be an EIGRP issue.

If you check the routing table on a spoke, who is the next-hop for the other spoke's subnet? I assume the Hub, but should be the other spoke.

There's a command on EIGRP to avoid that behavior:

EIGRP will, by default, set the IP next-hop to be the hub router for routes that it is advertising, even when advertising those routes back out the same interface where it learned them. So in this case, you need the following configuration command to instruct EIGRP to use the original IP next-hop when advertising these routes.

no ip next-hop-self eigrp 

Federico.

Hello Guys,

I have a pretty similar problem, I did setup a dual dmvpn topology with hsrp between the two hubs. a "sh crypto isakmp sa" command on the spoke also shows two ipsec vpn-connection, but the spoke only learns one route to the primary hub router through eigrp.

Here ist my Post regarding this problem: https://supportforums.cisco.com/message/3212901#3212901

is it possible to get your lab or configuration files, as I realy need to find a solution for this ?

Thank you.

Greetings, Thomas