Can't ping across IPSEC protected GRE tunnel with multiple VRFs

[nfrick]

Hello,

I was hoping someone else has done this sort of configuration and can help me.  I have configured a GRE tunnel with IPSEC tunnel protection between 2921 15.0(1r)M16 and 2951 15.0(1r)M16 routers.  When I remove IPSEC from the tunnel I can ping each destination from either router, so I know GRE is set up correctly.  However as soon as I add IPSEC I can no longer ping each side.  The ISAKMP SA state on each router is QM_IDLE and each has an active IPSEC inbound esp sa and outbound esp sa.

I've included the relevant config components off each router along with the results of show crypto session for the tunnel interface.  I have done debugging on each side for crypto isakmp, crypto ipsec, and tunnel protection but don't see any obvious issues.  I'm not sure if it is a VRF issue or not since I can ping when the tunnel protection is disabled.  What am I missing here?  Any help anyone can give me would be very welcome.

Router 1:

ip vrf TW

 rd 203:2

!

ip vrf p2p

 rd 101:1

crypto keyring TW vrf TW 

  pre-shared-key address 64.128.130.202 key

crypto isakmp policy 10

 encr aes 256

 hash md5

 authentication pre-share

 group 2

!

crypto isakmp policy 20

 encr aes 256

 hash md5

 authentication pre-share

crypto isakmp nat keepalive 20

crypto isakmp profile TW

   vrf TW

   keyring TW

   match identity address 64.128.130.202 255.255.255.255 TW

   keepalive 20 retry 2

   local-address GigabitEthernet0/0/0

crypto ipsec transform-set p2pstrong esp-3des esp-md5-hmac 

 mode transport

crypto ipsec fragmentation after-encryption

crypto ipsec profile p2psecure

 set security-association lifetime seconds 86400

 set transform-set p2pstrong 

 set isakmp-profile TW

interface Tunnel1

 bandwidth 20000

 ip vrf forwarding p2p

 ip address 10.0.4.1 255.255.254.0

 no ip unreachables

 no ip proxy-arp

 ip mtu 1400

 ip virtual-reassembly in

 ip tcp adjust-mss 1360

 ip policy route-map TW-set-sourced

 tunnel source GigabitEthernet0/0/0

 tunnel destination 64.128.130.202

 tunnel vrf TW

 tunnel protection ipsec profile p2psecure

interface GigabitEthernet0/0/0

 ip vrf forwarding TW

 ip address 64.128.71.86 255.255.255.252

 ip virtual-reassembly in

 duplex auto

 speed auto

ip route vrf TW 0.0.0.0 0.0.0.0 64.128.71.85

ip route vrf p2p 10.0.4.2 255.255.255.255 Tunnel1

Interface: Tunnel1

Profile: TW

Session status: UP-ACTIVE     

Peer: 64.128.130.202 port 500 

  IKEv1 SA: local 64.128.71.86/500 remote 64.128.130.202/500 Active 

  IPSEC FLOW: permit 47 host 64.128.71.86 host 64.128.130.202 

        Active SAs: 2, origin: crypto map

Router 2:

ip vrf TW

 rd 201:1

!

ip vrf p2p

 rd 202:3

crypto keyring TW vrf TW 

  pre-shared-key address 64.128.71.86 key

crypto isakmp policy 10

 encr aes 256

 hash md5

 authentication pre-share

 group 2

!

crypto isakmp policy 20

 encr aes 256

 hash md5

 authentication pre-share

crypto isakmp keepalive 20 3

crypto isakmp nat keepalive 20

crypto isakmp profile test2

   vrf global

   keyring TW

   match identity address 64.128.71.86 255.255.255.255 TW

   local-address GigabitEthernet0/2

crypto ipsec transform-set p2pstrong esp-3des esp-md5-hmac 

 mode transport

crypto ipsec profile p2psecure

 set security-association lifetime seconds 86400

 set transform-set p2pstrong 

 set isakmp-profile test2

interface Tunnel1

 bandwidth 20000

 ip vrf forwarding p2p

 ip address 10.0.4.2 255.255.254.0

 no ip unreachables

 no ip proxy-arp

 ip mtu 1400

 ip virtual-reassembly in

 ip tcp adjust-mss 1360

 tunnel source GigabitEthernet0/2

 tunnel destination 64.128.71.86

 tunnel vrf TW

 tunnel protection ipsec profile p2psecure

interface GigabitEthernet0/2

 ip vrf forwarding TW

 ip address 64.128.130.202 255.255.255.252

 ip virtual-reassembly in

 duplex auto

 speed auto

Interface: Tunnel1

Profile: test2

Session status: UP-ACTIVE     

Peer: 64.128.71.86 port 500 

  Session ID: 0  

  IKEv1 SA: local 64.128.130.202/500 remote 64.128.71.86/500 Active 

  IPSEC FLOW: permit 47 host 64.128.130.202 host 64.128.71.86 

        Active SAs: 2, origin: crypto map

[Pedro Lereno]

Hi,

Did you ping inside vrf TW?

What is the result of debug ip icmp?

Regards,

Pedro Lereno

[nfrick]

Thanks for replying, I am pinging within the p2p vrf.  I can ping the tunnel source and destination (eg. public IP addresses) on the TW vrf.  Here are the results of pinging each tunnel IP address from router 2 on both vrfs.  I had debug ip icmp on for all of these, the only entries are from ping vrf p2p 10.0.4.2 (which is this router).

VHD-FL-Spoke#ping vrf TW 10.0.4.2

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.4.2, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

VHD-FL-Spoke#ping vrf TW 10.0.4.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.4.1, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

VHD-FL-Spoke#ping vrf p2p 10.0.4.2

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.4.2, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

VHD-FL-Spoke#ping vrf p2p 10.0.4.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.4.1, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

*Sep 10 17:47:26.996: ICMP: echo reply sent, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:26.996: ICMP: echo reply rcvd, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:26.996: ICMP: echo reply sent, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:26.996: ICMP: echo reply rcvd, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply sent, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply rcvd, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply sent, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply rcvd, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply sent, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

*Sep 10 17:47:27.000: ICMP: echo reply rcvd, src 10.0.4.2, dst 10.0.4.2, topology BASE, dscp 0 topoid 4

Edit: If I disable IPSEC, I can successfully do a ping vrf p2p 10.0.4.1 from this router.  But I need IPSEC on.

[Pedro Lereno · Accepted solution]

Hi,

Let me see if I have understand:

The inside vrf is the p2p   10.0.4.1 -> 10.0.4.2 (inside packet of ipsec)

The front door vrf is TW  64.128.71.86 -> 64.128.130.202 (outside encrypted traffic)

You can "ping vrf TW 64.128.130.202" , that is OK!

You can not "ping vrf TW 10.0.4.2", that is also OK!

Try this:

Remove the static:

ip route vrf p2p 10.0.4.2 255.255.255.255 Tunnel1

(as 10.0.4.2 is a direct access on Tunnel you do not need a route)

and add:

ip route vrf p2p 0.0.0.0 0.0.0.0 10.0.4.2 (router 1)

ip route vrf p2p 0.0.0.0 0.0.0.0 10.0.4.1 (router 2)

Try to ping some hosts on p2p vrf

And also:

try remove "vrf TW" on "crypto isakmp profile TW" (router 1)

and "vrf global" on "crypto isakmp profile test2" (router 2)

Regards,

Pedro Lereno

[nfrick]

Pedro,

Thank you very much for your help.  I've made these changes and everything is working perfectly now.