Troubles using VRF-aware IPsec w/ crypto maps

tgrundbacher · ‎10-08-2013

I'm trying to get a lab setup to work with a C2951 (15.2(4)M4) peering with an ASA 5510 (9.1(2)). The config is based on crypto maps, since I want the C2951 to be the initiating side, and as far as I understand, VTIs wouldn't be working together with the ASA due to the default 'any' crypto statements that are being applied on SVTIs.

So I've set up this IKEv1-, crypto map-based lab, and the tunnel strictly won't come up; it seems that crypto doesn't find any interesting traffic at all (no debug crypto isakmp output pops up).

What I'm doing for testing is issuing a VRF Ping from a loopback interface of the C2951. I was following the following cheat sheet to configure the IOS box:

https://supportforums.cisco.com/docs/DOC-13524

Please see the attached config files and the setup drawing.

This is the way I'm testing it:

C2951#sh deb

Cryptographic Subsystem:

Crypto ISAKMP debugging is on

C2951#

C2951#ping vrf test 10.0.0.1 source lo 1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:

Packet sent with a source address of 40.0.0.1

.....

Success rate is 0 percent (0/5)

C2951#

Any hints for me, please?

Jouni Forss · ‎10-08-2013

Hi,

We have a similiar setup done with an ASR1000. We first tried with the 2x VRF setup (external and internal VRF) and it just wouldnt work at all. The 1 VRF setup (Customer VRF) seems to work just fine while the WAN link uses the global routing table.

I dont think we have any "ip route vrf" configurations that point towards the WAN link. I think on that part the default route configuration on the global routing table should be enough.

Also I am not sure if the "crypto isakmp policy" has any default setting for all the parameters. You could try defining the "hash sha" inside the policy.

Also I dont see a "set transform-set" configuration under the "crypto map" configurations.

I dont think you need the "crypto isakmp key" configuration as you have configured a "keyring" that is in use.

- Jouni

Jouni Forss · ‎10-08-2013

Also,

Your Phase 1 Policys dont match on the ASA and Router. The lifetimes are different.

- Jouni

tgrundbacher · ‎10-08-2013

Hi Jouni

Thanks for your reply. The ISAKMP policy has 'hash sha' configured as a default:

C2951#sh run all | s crypto isakmp policy

crypto isakmp policy 10

encr aes 256

hash sha

authentication pre-share

group 5

lifetime 28800

As for the transform set, there's a 'crypto ipsec transform-set default esp-aes esp-sha-hmac' in the config which should be used by the crypto map by default. Anyway, I've configured it individually to rule out any issues:

crypto ipsec transform-set AES-SHA esp-aes esp-sha-hmac

crypto map OUR-MAP 10 ipsec-isakmp

set peer 20.0.0.1

set transform-set AES-SHA

set pfs group5

set isakmp-profile 20.0.0.1

match address TEST1

reverse-route

No success so far.

I agree with you about the 'crypto isakmp key', it's just a leftover since this was my first trial; I've changed the config at some point and included the keyring section.

I'm not sure about the route but I believe we have to direct the traffic somewhere (towards the Internet) seen from the VRF perspective, so I believe a VRF route is mandatory. Still, the Ping wouldn't trigger any crypto process, the router just remains silent.

Regarding your following post about the ISAKMP lifetime; this is a parameter that doesn't have to match, since both boxes will negotiate it and agree upon the lower value. Anyway, I don't even get to the point where ISAKMP will start to negotiate, so there must be another problem (like routing?) that is the issue here...

Marcin Latosiewicz · ‎10-09-2013

If you're not getting any debugs that (tyically) is caused by two things:

1) Traffic is not hitting your crypto map (most likely due to leaking being misconfigured in your case).

2) For whatever reason locally switched traffic does fail to do route lookup.

a) My suggestion use "global" keyword when configuring static routing

http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_pi/command/iri-cr-a1.html#GUID-24399FED-0F6A-479C-B88A-D7FB74257058

b) rely on RRI to configure your VRF-aware IPsec.

This example is overly simplified but you will get the idea:

http://www.cisco.com/en/US/docs/ios-xml/ios/sec_conn_vpnav/configuration/15-mt/sec-rev-rte-inject.html#GUID-DEBFE993-16DF-4599-946A-1B7A42521C92

tgrundbacher · ‎10-09-2013

Hi Marcin

Thanks for your input. I agree with your diagnosis. This is what I've done meanwhile:

crypto map OUR-MAP 10 ipsec-isakmp

set peer 20.0.0.1

set transform-set AES-SHA

set pfs group5

set isakmp-profile 20.0.0.1

match address TEST1

reverse-route remote-peer 20.0.0.1

ip route 0.0.0.0 0.0.0.0 30.0.0.1

ip route vrf test 10.0.0.0 255.255.255.0 GigabitEthernet0/0 30.0.0.1

ip route vrf test 10.0.0.0 255.255.255.0 30.0.0.1 global

(I've used all routing statements together, as well as the the top one combined with the last two alternately.)

As I've also been playing around w/ IKEv2 before and left these commands on the box, I thought it might be a good idea to clear the config, boot the box and re-apply the IKEv1 config. Still, the problem persists - the VRF Ping doesn't trigger any crypto process.

tgrundbacher · ‎10-09-2013

OK, I've forgot to no shut the gi0/0 interface after the boot and re-application of the config. Now IKEv1 gets triggered! They can't agree upon some parameters right now, but at least I can continue to troubleshoot! Thanks so far!

Marcin Latosiewicz · ‎10-09-2013

Good, once you're at that point you're not far from success.

tgrundbacher · ‎10-09-2013

Marcin

Meanwhile the tunnel's up, and I've found out the optimal way to configure the VPN routing so the VRF knows where to route its packets to. I was able to do this by removing any 'ip route vrf ...' and adding the 'reverse-route remote-peer static' command under the crypto map section (see below). However, I now have the problem that only every second Ping gets answered when I issue the VRF Ping, yet all Pings get answered when I initiate the Pings from the remote side (src 10.0.0.1). What do you think might be the cause for this?

As a reference, see my findings by playing around with the reverse-route command:

C2951(config-crypto-map)#reverse-route remote-peer 20.0.0.1 gateway

C2951(config-crypto-map)#do sh ip route vrf test

Routing Table: test

Codes: -snip-

Gateway of last resort is not set

-> no route to 10.0.0.0 /24 appears

40.0.0.0/32 is subnetted, 1 subnets

C 40.0.0.1 is directly connected, Loopback1

C2951(config-crypto-map)#reverse-route static

C2951(config-crypto-map)#do sh ip route vrf test

Routing Table: test

Codes: -snip-

Gateway of last resort is not set

10.0.0.0/24 is subnetted, 1 subnets

S 10.0.0.0 [1/0] via 20.0.0.1

40.0.0.0/32 is subnetted, 1 subnets

C 40.0.0.1 is directly connected, Loopback1

C2951(config-crypto-map)#reverse-route remote-peer 20.0.0.1 static

C2951(config-crypto-map)#do sh ip route vrf test

Routing Table: test

Codes: -snip-

Gateway of last resort is not set

10.0.0.0/24 is subnetted, 1 subnets

S 10.0.0.0 [1/0] via 20.0.0.1, GigabitEthernet0/0

40.0.0.0/32 is subnetted, 1 subnets

C 40.0.0.1 is directly connected, Loopback1

C2951#ping vrf test 10.0.0.1 sou lo1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:

Packet sent with a source address of 40.0.0.1

.!.!.

Success rate is 40 percent (2/5), round-trip min/avg/max = 1/1/1 ms

C2651XM#ping 40.0.0.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 40.0.0.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

Marcin Latosiewicz · ‎10-09-2013

Clear your routing table. If you left the two 10.0.0.0/24 VRF routes you're most likely doing per-packet load balancing.

tgrundbacher · ‎10-09-2013

There are no VRF routes left in the config, and I've cleared the global and the VRF routing table. Even rebooted the box. Still only half of the Pings get answered. There are no crypto ipsec errors, so it should have something to do with routing...but what?

C2951#sh crypto ipsec sa

interface: GigabitEthernet0/0

Crypto map tag: OUR-MAP, local addr 30.0.0.2

protected vrf: test

local ident (addr/mask/prot/port): (40.0.0.1/255.255.255.255/0/0)

remote ident (addr/mask/prot/port): (10.0.0.0/255.255.255.0/0/0)

current_peer 20.0.0.1 port 500

PERMIT, flags={origin_is_acl,}

#pkts encaps: 14, #pkts encrypt: 14, #pkts digest: 14

#pkts decaps: 8, #pkts decrypt: 8, #pkts verify: 8

#pkts compressed: 0, #pkts decompressed: 0

#pkts not compressed: 0, #pkts compr. failed: 0

#pkts not decompressed: 0, #pkts decompress failed: 0

#send errors 0, #recv errors 0

local crypto endpt.: 30.0.0.2, remote crypto endpt.: 20.0.0.1

path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0

current outbound spi: 0xEB02ACDA(3942821082)

PFS (Y/N): Y, DH group: group5

inbound esp sas:

spi: 0x1A943A9F(445921951)

transform: esp-aes esp-sha-hmac ,

in use settings ={Tunnel, }

conn id: 18009, flow_id: ISM VPN:9, sibling_flags 80000040, crypto map: OUR-MAP

sa timing: remaining key lifetime (k/sec): (4225929/3571)

IV size: 16 bytes

replay detection support: Y

Status: ACTIVE(ACTIVE)

inbound ah sas:

inbound pcp sas:

outbound esp sas:

spi: 0xEB02ACDA(3942821082)

transform: esp-aes esp-sha-hmac ,

in use settings ={Tunnel, }

conn id: 18010, flow_id: ISM VPN:10, sibling_flags 80000040, crypto map: OUR-MAP

sa timing: remaining key lifetime (k/sec): (4225928/3571)

IV size: 16 bytes

replay detection support: Y

Status: ACTIVE(ACTIVE)

outbound ah sas:

outbound pcp sas:

C2951#sh ip route 10.0.0.0

% Network not in table

C2951#sh ip route vrf test 10.0.0.0

Routing Table: test

Routing entry for 10.0.0.0/24, 1 known subnets

S 10.0.0.0 [1/0] via 20.0.0.1, GigabitEthernet0/0

tgrundbacher · ‎11-21-2013

After involving TAC it seems to be a bug (to be confirmed by DE) in the ASA code. The problem for the 50% success rate of Pings through the VPN tunnel is caused by the enabled ICMP inspect. If I disable the inspect, I get a 100% success rate. The issue can be reproduced on 9.1(2) and 9.1(3), yet not on 9.1(1).

tgrundbacher · ‎11-24-2013

Bug ID CSCul55863