Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

DMVPN on Spoke Troubleshooting Order and Component Responsibilities

 

Disclaimer: This content has been initially created by Alex Honore, Graham Barlet, Raffaele Brancaleoni from Cisco.

 

Introduction.

This document is indetnded to show troubleshooting order of operation when troubleshooting DMVPN setup.

 

It will also discuss responsibilites of different components, but not in detail.

 

It is intended as a troubleshooting framework and a referance on which commands to be used.

 

Before you start.

Please remember, that you should always start troubleshooting of DMVPN on spokes and not hubs.

Impact of changes on spokes is limited to particular location, while changes/troubleshooting on hub side are potetially impacting entire DMVPN cloud.

 

Using a recent revision of IOS software helps overcome a lot of problems with monitoring and troubleshooting.

 

Add this command to running configuration on both sides of problematic tunnel:

 crypto logging session 

allows to see indepth information about failures and potential reasons.

 

Order of troubleshooting

This section describes step by step approach to troubleshooting DMVPN.

Using this framework allows to easily verify offending component and apply remediation.

NHRP

NHRP is the trigger for spoke to hub initial tunnel establishment.

 

Useful commands.

 

Key components at this stage can be verfied in the configuration:

show run interface tunnel X

Things to verify:

 

  • Spoke: contains a static NHRP mapping indicating that hubs' tunnel interface IP is behind a particular NBMA address
  • Spoke:NHS should point to hubs' tunnel interface IP address.
  • Spoke: If multicast is needed - multicast mapping for NBMA address of hub(s).
  • Spoke and hub: NHRP network id is configured.
  • Hub: If multicast is needed - multicast dynamic map.

 

 

IKE

IKE provides keying material to IPsec and is therefor a preamble to it.

It is during IKE negotiatin that calls are made to PKI infrastructure.

 

Useful commands.

Understand what state IKE is at the moment.

show crypto isakmp sa

MM_NO_STATE will mean a failure or expired SA.

MM_KEY_EXCH will signifiy that we are in process or stuck during authentication (PKI or pre-shared).

 

PKI

PKI is component responsible for the authentication of peers in IKE negotiation.

To accomplish its task, it requites the retrieval and processing of revocation information (CRL).

 

Useful commands.

Check the time:

sh clock 

Check the validity of certificates on both sides:

show crypto pki cert

Check if you have a recent version of CRL downloaded:

show cry pki crls

 

Check availability of CDP:

telnet IP_OR_HOSTNAME 80

  

     GET /<name_of_crl_file> HTTP/1.0

 

IPsec

IPsec by itself is stateless, but encrypted traffic emission and reception capabilities must be confirmed (think about IP protocol type 50/51 filtering in transport network for ex.)

 

Useful commands.

Verfies if both IKE and IPsec are up.

show crypto session [detail]

For working session you will see TWO active SAs per peer. One SA for inbound traffic and one SA for outbound traffic.

 

Verify if tunnel is passing traffic.

show crypto ipsec sa peer IP

Working scenario - encaps and decaps section counters are non-zero AND increasing.

 

GRE

 

Provide multiprotocol support capability & multicast support to IPsec along with simplifying configs. Only fails if tunnel keys between GRE endpoints are different

 

Useful commands.

On both hub and spoke

show tunnel endpoint tunnel X

On spoke this information should be populated by static NHRP mapping.

NHRP (again!)

 

  NHRP performs the initial registration to Hub which forms the basis of DMVPN and is at the root of the spoke to spoke dynamic IPsec tunnels construction

 

Useful commands.

On spoke, check if you see responses from NHS.

show ip nhrp nhs

 

On hub , verify if dynamic NHRP entiries were populated.

show ip nhrp tunnel X

show ip nhrp multicast

 

Routing protocol

Propagate the necessary routing information between hubs and spokes  

Useful commands.

It's hard to point to one set of commands to troubleshoot this - DMVPN supports multiple routing protocols.

In any case you need to verify if a particular adjacancy/neighborship came up correctly and whether you are sending and receiving correct prefixes.

 

BGP

BGP is by far the best protocol to scale in DMVPN networks. With recent additions to it (like dynamic listener) we are able to create working configuration hassle-free.

 

  • Verify if neighborship is up

show ip bgp [vpnv4 vrf VRF_NAME] summary

  • Verify which prefixes you have in BGP table and from whom

show ip bgp [vpnv4 vrf VRF_NAME]

 

  • Verify what prefixes you are receiving from particular neighbor

show  ip bgp [vpnv4 vrf VRF_NAME] neighbor IP_ADDRESS routes

  • Verify what prefixes you are sending to particular neighbor.

show  ip bgp [vpnv4 vrf VRF_NAME] neighbor IP_ADDRESS advertised

  • Verify what prefixes made it into routing table.
show ip router bgp

 

 

EIGRP

 

  • Verify that neighborship is up
show ip eigrp neighbor

 

  • Verify the topology table to see what prefixes are inserted.

 

show ip eigrp topology

 

 

  • Verify what prefixes made it into routing table.

 

show ip router eigrp
Comments

Thanks for this great contribution Marcin!

Cisco Employee

Extremely easy to digest treatment of a relatively complex topic. Thanks!!!

14510
Views
10
Helpful
2
Comments