01-19-2015 06:50 AM
We are using the DMVPN_IPSEC HUB and spoke Tech. We have many peers connected in HUB without any issue. One of the Peer BGP is not established.
1) Verified tunnel configuration (DMVPN MAP) no issue on that, below are the items verified in spoke end
2) Checked ISP, no issue identified we are able to trace the HUB and physical ip without any issue and ICMP blocked in HUB ISP interface. Only esp and gre are permitted.
3) Crypto configuration working fine .Phase 1 up and working fine but phase 2 decryption not happening since the GRE tunnel is down. Below is the logs
Router#sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
x.x.x.x x.x.x.x QM_IDLE 1300 ACTIVE
x.x.x.x x.x.x.x QM_IDLE 1299 ACTIVE
Router##sh crypto ipsec sa
#pkts encaps: 1277, #pkts encrypt: 1277, #pkts digest: 1277
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
4)Checked DMVPN sockets has been opened , Yes
5)checked crypto session UP_ACTIVE
5)DMVPN output status showing as “NHRP” , It suppose to be “UP”
Router #sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==============================
Interface: Tunnelxxx, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 x.x.x.x x.x.x.x NHRP 6d00h S (state should up)
1 x.x.x..x x.x.x..x NHRP 6d00h S
6) Enabled debug “DMVPN ALL ALL”
Jan 19 08:29:06.926 EST: NHRP: NHRP successfully resolved x.x.x.x to NBMA x.x.x.x
Jan 19 08:29:10.926 EST: NHRP: NHRP successfully resolved x.x.x.x to NBMA x.x.x.x
Jan 19 08:29:18.926 EST: NHRP: NHRP successfully resolved x.x.x.x to NBMA x.x.x.x
Jan 19 08:29:19.262 EST: NHRP: NHRP successfully resolved x.x.x.x to NBMA x.x.x.x
Jan 19 08:29:54.474 EST: ISAKMP (1299): received packet from x.x.x.x dport 500 sport 500 INTERNET (I) QM_IDLE
Jan 19 08:30:43.842 EST: IPSEC(crypto_map_check_
7)BGP status showing as below
Router #sh ip bgp summary
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
x.x.x.x 4 64609 0 0 1 0 0 never Active
x.x.x.x 4 64609 0 0 1 0 0 never Idle
Router #
01-19-2015 11:05 AM
"mtree says we have SA but couldn't find current outbound SA" that one is peculiar would indicate some problem at crypto map (not the config necessarily) level.
#pkts encaps: 1277, #pkts encrypt: 1277, #pkts digest: 1277
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
Probably connected but with that in place you will never receive NHRP registration reply. (Assuming it's from a spoke).
Removing crypto or whole tunnel config might fix this, but you might also consider an upgrade and or opening a TAC case to verify this.
01-19-2015 08:30 PM
Hi Marchin,
Thanks , I removed tunnel configuration and i reconfigured still the DMVPN tunnel doesn't come up.
Previously it was working since last week we are facing this issue.
01-19-2015 11:27 PM
I'd check that with TAC, troubleshooting over forums takes lots and lots of time and we'd need outputs you're not willing to share details of (IP addresses etc).
You might also try a sniffer trace on the external interface - will show you whether you're really sending and really receiving encrypted. (Check also dynamic NHRP mapping on hub).
Still seeing this after removing tunnel configs?
mtree says we have SA but couldn't find current outbound SA.
01-19-2015 11:36 PM
Aftyer removing tunnel configuration means; should i remove all related nhrp or Tunnel protection configuration or completed Tunnel configuration ?
I have removed tunnel configuration and reconfigured all the configuration.
Could you please tell me clearly so that i can implement and check
01-19-2015 11:51 PM
Let me elaborate and take a step back.
1) When you configure DMVPN tunnel IKE and IPsec SAs are negotiated between hub and spoke (spoke knows who to negotiate with based on NHRP config).
You should have IPsec SA DB entry with two active IPsec SAs (inbound and outbound).
That mtree error is suggesting you might not, but it also might be unrelated to the entire config... it depends :-)
Removing applied tunnel protection and crypto maps from all interfaces might fix this.
2) When IPsec SAs are up spoke sends NHRP registration to spoke (it's encrypted so it will trigger encaps to go up).
On hub side you should check whether for this spoke's IP address you're seeing that same packet decaps'ulate .... show crypto ipsec sa peer x.x.x.x
3) When NHRP registration request is received hub will create a dynamic NHRP entry in it's NHRP table - check whether it exists and that it points to right NBMA.
4) NHRP Registration reply is sent to address based on NHRP mapping on hub and should trigger encaps on IPsec SA for this peer. Check that it's happening.
There's a chance your encrypted packets are dropped in transit, looking at the IPsec SA output you punted, but there are a few things to check.
01-20-2015 01:20 AM
Hi Marchin,
1st option which u suggested , i tried and no changes still the same & we dont have any cryptomap for intresting traffic because we are using dynamic VTI interface(tunnel protection) so the intresting traffic is source ip and destination ip gre packet permitted
2nd option , In HUB end i am not getting anything to this particular spoke (enc & dec 0 0 )
But phase 1 tunnel up , phase 2 shows as below because gre tunnel not up
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
01-20-2015 01:36 AM
Even though you're do not have crypto map configured, crypto map (db) is enabled - it is how the logical construct is called.
You can do "show crypto map" in DMVPN deployment and see for yourself.
"tunnel protection ..." will add crypto map db entry.
VTI != DVMPN, VTI = mode ipsec ipv4/6 , DMVPN = gre over IPv4/IPv6 with or without IPsec.
IF you have IPsec SAs and you don't see decaps on hub there's your problem.
Encrypted packets are not making it to the hub. You can verify by sniffing traffic on both ends.
small edit:
phase 2 shows as below because gre tunnel not up
Just to be clear, DMVPN is GRE over IPsec not IPsec over GRE, GRE will be up as long as a few basic config criteria are met (for example destination IP reachable), on top we're talking about mgre, so even a bit different.
You establish IPsec, on top of IPsec SAs rides GRE, on top of that you have your NHRP. The output showing 0 decaps and 0 encaps means nothing was received or sent by IPsec.
01-21-2015 04:32 AM
Could you please explain below
1)What is the difference between GRE over IPSEC vs IPSEC over GRE?
2)In DMVPN which has to be come up first GRE tunnel or IPSEC tunnel ?
I have uploaded new ios in the box still the BGP session not coming up
01-21-2015 09:55 AM
DMVPN is GRE over IPsec, in which case GRE will not flow until its transport layer (IPsec) is not established.
IPsec over GRE is very rare case ...in which case you have IPsec running over a GRE tunnel.
I don't think upgrade would help, what you've shown so far is indicating that spoke is sending encrypted packets and hub is never receiving them - indicating a problem in the path.
But as I said feel free to evaluate this with a sniffer trace.
01-22-2015 04:04 AM
Thanks for teh explanation
I’m trying to understand one thing since long back below is
1) GRE over IPSEC; u said in DMVPN will use, which tunnel gets form first GRE or IPSEC?
2) IPSEC over GRE where people are using?
3) Once GRE established and we formed the spoke to spoke tunnel as well, once the NHRP tunnel expires (On demand Tunnel), how the re-negotiation would be
> First IPSEC negotiation or NHRP registration
01-22-2015 04:35 AM
IPsec is formed first, if you do not have active IPsec SAs GRE cannot flow.
IPsec over GRE I've seen only with GetVPN, and when GRE was an access method.
If your NHRP entry (not tunnel) expires you're missing information on who the end point for IPsec tunnel is. Depending on the config, this might cause traffic to go through hub or be dropped.
01-22-2015 07:21 PM
So as per your explanation GRE over IPSEC , First IPSEC tunnel then the GRE tunnel
Can u please give me any best link which can explain the packet level , because as per my understand also as like yours , but people over here telling GRE should up then IPSEC.
In previous note , i raised the question but i missed some words here below
Once GRE established and we formed the spoke to spoke tunnel for the particular subet in differnt location, once the NHRP tunnel expires (On demand Tunnel) for the particular subnet , how the re-negotiation would be?
Again IPSEC negotiation would be first for the new subnet which is hosted in remote location or directly NHRP tunnel will form
01-23-2015 01:10 AM
I think there may be some confusion on their part, but indeed IPsec is optional in DMVPN, in which case you would see GRE packets in the wild.
But maybe it should be mentioned here, if the GRE tunnel _interface_ is down there will not be any IKE sent.
There is no such thing as NHRP tunnel (at least that I'm aware of).
What you have is a NHRP resolution process, have a look at BRKSEC-4054 presentation, it outlines NHRP resolution request and reply depending on design (phase 2 or phase 3).
There's a decent packet decode picture here
http://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/WAN_and_MAN/VPNLoad/VPN_Load/ch8_Dual.html#wp1038732
showing GRE encapsulated within IPsec.
You will also find a plenty of information from Cisco Live slides and CCIE training sites.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: