Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Webcast-Catalyst9k
New Member

Understanding DMVPN Issues

My DMVPN spoke router can PING the DMVPN hub, but it looks like the traffic is going over the Internet and not the GRE tunnel.

COBOU27THBASELINE_RL01#ping 138.69.156.3

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 138.69.156.3, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 72/80/96 ms

COBOU27THBASELINE_RL01#

The tunnel is up.

COBOU27THBASELINE_RL01#sh crypto isa sa

dst src state conn-id slot status

138.69.156.3 173.8.255.109 QM_IDLE 37 0 ACTIVE

138.69.152.3 173.8.255.109 QM_IDLE 38 0 ACTIVE

COBOU27THBASELINE_RL01#

COBOU27THBASELINE_RL01#trace 138.69.156.3

Type escape sequence to abort.

Tracing the route to 138.69.156.3

1 * * *

2 68.86.105.145 16 msec 16 msec 16 msec

3 68.86.103.122 20 msec 24 msec 16 msec

4 68.86.179.97 52 msec 36 msec 16 msec

5 68.86.91.1 24 msec 24 msec 32 msec

6 68.86.85.249 60 msec 60 msec 48 msec

7 65.19.100.165 68 msec 44 msec 48 msec

8 64.15.0.97 64 msec 108 msec 72 msec

9 64.15.0.194 72 msec 124 msec 80 msec

10 64.15.8.30 152 msec 72 msec 76 msec

11 64.15.8.98 140 msec 80 msec 100 msec

12 65.19.121.134 92 msec 72 msec 80 msec

13 65.51.99.170 72 msec 72 msec 76 msec

14 10.40.255.19 !A * !A <--That is the DMVPN HUB

COBOU27THBASELINE_RL01#

COBOU27THBASELINE_RL01#sh ip ro 138.69.156.3

Routing entry for 138.69.152.0/21

Known via "static", distance 1, metric 0

Routing Descriptor Blocks:

* 173.8.255.110

Route metric is 0, traffic share count is 1

COBOU27THBASELINE_RL01#

Anyone?

1 ACCEPTED SOLUTION

Accepted Solutions
Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

I had worked for some years on MPLS networks and in MPLS there is a similar scenario with a difference:

there is a command that decides if intermediate MPLS router hops can be seen or not when starting a traceroute from a Customer Edge router.

with

mpls ip propagate-ttl

the traceroute shows intermediate hops

with

no mpls ip propagate-ttl

intermediate hops are hidden.

The action of the command is to copy the inner ip packet TTL in the MPLS header TTL field when enabled.

So I think that the same happens with ip packets encapsulated in GRE headers.

I don't see any other possible explanation of the observed behaviour.

Hope to help

Giuseppe

10 REPLIES
Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

to perform the tests use a ping with a source = internal lan and destination internal lan on hub router.

I remember I had similar issues for spoke to spoke tunnels that were not triggered unless pinging from lan to lan.

Also if 138.69.156.3 is the public address it is correct it is routed out of the tunnel.

you need to see some routes via the mGRE tunnel interface using an IGP like EIGRP or OSPF is typical.

see

http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/DMVPN_1.html

Hope to help

Giuseppe

New Member

Re: Understanding DMVPN Issues

Hi, Giuseppe:

Im not sure you understood what I was asking.

The spoke router has a tunnel interface, a vlan (users LAN) interface, and the public address interface to the SP.

The hub router has the public address interface to the SP, the tunnel interface, etc.

From the spoke, I do NOT source the LAN subnet when I run the PING. I just PING the outside interface of the hub (the tunnel endpoint), and I go over the Internet. I do not use the tunnel, as the trace shows. On the other hand, if I trace to something that sits behind the outside interface of the hub, it will use the tunnel, even though I am still not sourcing the LAN. Im just running a straight trace.

COBOU27THBASELINE_RL01#trace 138.69.21.250 <----sits way behind the hub on the internal LAN of the data center.

Type escape sequence to abort.

Tracing the route to 138.69.21.250

1 10.40.16.1 84 msec 80 msec 84 msec <----tunnel on hub, then it goes to a FW, so we lose the trace.

2 * * *

3 * * *

4 * * *

5

COBOU27THBASELINE_RL01#ping 138.69.21.250

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 138.69.21.250, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 84/86/92 ms

COBOU27THBASELINE_RL01#

My first question is why this behavior?

Thanks

Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

I think that to understand what is happening you should attach

the configuration of the HUB and of the spokes (the parts of interest)

sh ip route

sh ip nhrp

on both ends

Are you using a dynamic routing protocol over the mGRE tunnel ?

Hope to help

Giuseppe

New Member

Re: Understanding DMVPN Issues

Giuseppe:

I think I'm starting to figure it out on my own. :-)

My first question was why the tunnel is not used when I trace from the spoke router to the outside, tunnel destination interface on the hub, without sourcing the user-LAN interface. I think the answer is actually clear. Dont know why I didnt see it before....

The route in the routing table of the spoke for the outside hub interface is a static route whose next hop points to the provider. Ex: ip route 138.60.X.X 255.255.255.255 .

So, given that the LAN interface is not being sourced, which means that the tunnel is not being used, the trace packet to the outside interface of the hub will adopt a source address of the exit interface and get routed like any other normal iP datagram. We are therefore able to see all the intervening hops. What I mean by the "tunnel is not being used" is that the packet does not go through any GRE encapsulation at the spoke -- and probably not even any IPSec (ESP) enapsulation either. This latter point I am not sure of, actually.

The second point I am not sure of is why when we use a tunnel for transport that we do not see the intervening hops when we run a trace. My "guess" is that the GRE and IPSec encapsulation cause the TTL counter to not get decremented as normally occurs during a trace. Typically, the TTL in the IP header is sent with a value of 1, and when it is returned by the first hop it is resent with a TTL of 2, and so on and so forth, until the final destination router responds "positively." This last point is something I would like to be able to read about but have not been able to find at all on the Internet.

Do you know of a white paper that details the step by step procedure for GRE encapsulation when it is run over IPSec? I mean a real detailed thing...?

What are your thoughts on what I have written so far?

Thanks

Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

I think you have found the answer by yourself.

by examining the IP routing table if you see a route like 138.60.x.x using as next hop the ISP provider this is correct:

the only requirement to have stable tunnels is that the ip addresses used as IPSec endpoints (or GRE endpoints if not using tunnel protection) the public ip addresses are not advertised over the tunnel itself.

So this static route is needed to be able to setup the IPSec SA that carries the mGRE tunnel inside.

Failing to do so causes recursive errors and tunnel flaps up/down.

So starting from this info when you trace without specifying a source the nearest interface to the destination is used for the source IP address.

In your case when you don't see the intermediate hops traffic is going inside ipsec/GRE/ip.

When you see the intermediate hops there is no GRE and no ipsec and packets are sent with no additional encapsulation.

I attach for you a presentation from networkers 2k3 that has the diagrams of encapsulation.

It misses some new features but it is very clean explanation of DMVPN

Hope to help

Giuseppe

New Member

Re: Understanding DMVPN Issues

Thanks, Giuseppe:

"In your case when you don't see the intermediate hops traffic is going inside ipsec/GRE/ip.

When you see the intermediate hops there is no GRE and no ipsec and packets are sent with no additional encapsulation."

I have established this. I was wondering why this is the case. It must have something to do with the TTL counter and how it's decremented when in an IPSec tunnel.

I havent read the document you sent me yet, but it seems OK. I would have liked to find something that has a narrative/dialogue with it. This is a slide presentation that is designed to have someone instructing to fill in the blanks. But it's better than nothing and I thank you for sending it.

Thanks

Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

about the TTL with encapsulated packets:

the TTL parameter is not copied in the external IPv4 header so when doing traceroute with TTL=1 the inner packet is examined at the router at the other end of the GRE tunnel for this reason you don't see the intermediate hops.

Hope to help

Giuseppe

New Member

Re: Understanding DMVPN Issues

Giuseppe:

This is precisely what I suspected. So, the INNER IPv4 header -- the original one that has been encapsulated by GRE -- does not export its TTL counter to the IPSec outer header (ESP header). Correct?

Where did you read this? I haven't been able to find it anywhere...

Thanks

Hall of Fame Super Silver

Re: Understanding DMVPN Issues

Hello Joe,

I had worked for some years on MPLS networks and in MPLS there is a similar scenario with a difference:

there is a command that decides if intermediate MPLS router hops can be seen or not when starting a traceroute from a Customer Edge router.

with

mpls ip propagate-ttl

the traceroute shows intermediate hops

with

no mpls ip propagate-ttl

intermediate hops are hidden.

The action of the command is to copy the inner ip packet TTL in the MPLS header TTL field when enabled.

So I think that the same happens with ip packets encapsulated in GRE headers.

I don't see any other possible explanation of the observed behaviour.

Hope to help

Giuseppe

New Member

Re: Understanding DMVPN Issues

Giuseppe:

NOW you're talkin', my man....

That does make sense.

Thanks for sticking with me on this one.

Have you noticed an interesting pattern?

I ask a question but resolve it myself, with you filling in some of the blanks. And no one else joins in our conversations. Hmmmm

Maybe we got a system you and I. :-)

Grazie.

405
Views
7
Helpful
10
Replies
CreatePlease to create content