Help with L2L VPN and inbound network NAT for their remote network not working.

[kpruett]

I'm having difficulty with NAT inbound for a remote VPN site-to-site network from site, "WIDGET," noted below. I have a site-to-site VPN with the following requirements that I need to ensure works with current NAT design and configuration. This isn't working correctly, as it is giving me logging pointing to the inbound NAT of their network not converting back when attempting to egress going to them.

Near side networks (my side):
10.52.80.0/22
10.52.16.0/21

Far side networks (WIDGET side):
192.168.253.144/28

What I need WIDGET's far side network to appear as internal to my side:
10.50.34.144/28

I need that NAT working for their network in both directions. They should appear to me as 10.50.34.144/28, but outbound traffic to them should be converted back to 192.168.253.144/28 so as to match the VPN SA.

What my NAT currently looks like for NONAT and inbound NAT:
object-group network WIDGET-FAR-NETWORKS
 description Far side vpn networks for WIDGET B2B VPN connection.
 network-object 192.168.253.144 255.255.255.240
object-group network WIDGET-NEAR-NETWORKS
 description near side vpn networks for WIDGET B2B VPN connection.
 network-object 10.52.16.0 255.255.248.0
 network-object 10.52.80.0 255.255.252.0
object-group network WIDGET-FAR-NAT
 description inbound network NAT for WIDGET B2B VPN connection.
 network-object 10.50.34.144 255.255.255.240

nat (inside,outside) source static WIDGET-NEAR-NETWORKS WIDGET-NEAR-NETWORKS destination static WIDGET-FAR-NAT WIDGET-FAR-NAT no-proxy-arp
nat (outside,inside) source static WIDGET-FAR-NETWORKS WIDGET-FAR-NAT

Logging pointing to the NAT between us not being two-way and thus not matching the SA:
5|Oct 03 2014|15:01:47|713904|||||IP = 100.100.100.2, Received encrypted packet with no matching SA, dropping
3|Oct 03 2014|15:01:47|113019|||||Group = 100.100.100.2, Username = 100.100.100.2, IP = 100.100.100.2, Session disconnected. Session Type: LAN-to-LAN, Duration: 0h:00m:00s, Bytes xmt: 0, Bytes rcv: 0, Reason: crypto map policy not found
5|Oct 03 2014|15:01:47|713259|||||Group = 100.100.100.2, IP = 100.100.100.2, Session is being torn down. Reason: crypto map policy not found
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE SENDING Message (msgid=dfaef5ae) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 80
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing qm hash payload
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing IKE delete payload
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing blank hash payload
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, sending delete/delete with reason message
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, IKE SA MM:dc9c25a7 terminating:  flags 0x0101c002, refcnt 0, tuncnt 0
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, IKE SA MM:dc9c25a7 rcv'd Terminate: state MM_ACTIVE  flags 0x0001c042, refcnt 1, tuncnt 0
3|Oct 03 2014|15:01:47|713902|||||Group = 100.100.100.2, IP = 100.100.100.2, Removing peer from correlator table failed, no match!
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, sending delete/delete with reason message
7|Oct 03 2014|15:01:47|715065|||||Group = 100.100.100.2, IP = 100.100.100.2, IKE QM Responder FSM error history (struct &0x7866ac60)  <state>, <event>:  QM_DONE, EV_ERROR-->QM_BLD_MSG2, EV_NEGO_SA-->QM_BLD_MSG2, EV_IS_REKEY-->QM_BLD_MSG2, EV_CONFIRM_SA-->QM_BLD_MSG2, EV_PROC_MSG-->QM_BLD_MSG2, EV_HASH_OK-->QM_BLD_MSG2, NullEvent-->QM_BLD_MSG2, EV_COMP_HASH
3|Oct 03 2014|15:01:47|713902|||||Group = 100.100.100.2, IP = 100.100.100.2, QM FSM error (P2 struct &0x7866ac60, mess id 0x99df2264)!
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE SENDING Message (msgid=fdbfbe63) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 464
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing qm hash payload
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing blank hash payload
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, sending notify message
3|Oct 03 2014|15:01:47|713061|||||Group = 100.100.100.2, IP = 100.100.100.2, Rejecting IPSec tunnel: no matching crypto map entry for remote proxy 10.50.34.240/255.255.255.240/0/0 local proxy 10.52.0.0/255.255.0.0/0/0 on interface outside
6|Oct 03 2014|15:01:47|713905|||||Group = 100.100.100.2, IP = 100.100.100.2, Skipping dynamic map SYSTEM_DEFAULT_CRYPTO_MAP sequence 65535: cannot match peerless map when peer found in previous map entry.
7|Oct 03 2014|15:01:47|713222|||||Group = 100.100.100.2, IP = 100.100.100.2, Static Crypto Map check, map = outside_map, seq = 1, ACL does not match proxy IDs src:10.50.34.240 dst:10.52.0.0
7|Oct 03 2014|15:01:47|713221|||||Group = 100.100.100.2, IP = 100.100.100.2, Static Crypto Map check, checking map = outside_map, seq = 1...
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, QM IsRekeyed old sa not found by addr
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing notify payload
7|Oct 03 2014|15:01:47|713034|||||Group = 100.100.100.2, IP = 100.100.100.2, Received local IP Proxy Subnet data in ID Payload:   Address 10.52.0.0, Mask 255.255.0.0, Protocol 0, Port 0
7|Oct 03 2014|15:01:47|714011|||||Group = 100.100.100.2, IP = 100.100.100.2, ID_IPV4_ADDR_SUBNET ID received--10.52.0.0--255.255.0.0
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing ID payload
7|Oct 03 2014|15:01:47|713035|||||Group = 100.100.100.2, IP = 100.100.100.2, Received remote IP Proxy Subnet data in ID Payload:   Address 10.50.34.240, Mask 255.255.255.240, Protocol 0, Port 0
7|Oct 03 2014|15:01:47|714011|||||Group = 100.100.100.2, IP = 100.100.100.2, ID_IPV4_ADDR_SUBNET ID received--10.50.34.240--255.255.255.240
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing ID payload
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, processing ISA_KE for PFS in phase 2
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing ke payload
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing nonce payload
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing SA payload
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing hash payload
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE RECEIVED Message (msgid=99df2264) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + KE (4) + ID (5) + ID (5) + NOTIFY (11) + NONE (0) total length : 404
7|Oct 03 2014|15:01:47|714003|||||IP = 100.100.100.2, IKE Responder starting QM: msg id = 99df2264
7|Oct 03 2014|15:01:47|715080|||||Group = 100.100.100.2, IP = 100.100.100.2, Starting P1 rekey timer: 82080 seconds.
7|Oct 03 2014|15:01:47|713121|||||IP = 100.100.100.2, Keep-alive type for this connection: DPD
3|Oct 03 2014|15:01:47|713119|||||Group = 100.100.100.2, IP = 100.100.100.2, PHASE 1 COMPLETED
6|Oct 03 2014|15:01:47|113009|||||AAA retrieved default group policy (B2B-VPN-POLICY) for user = 100.100.100.2
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) + NONE (0) total length : 96
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing dpd vid payload
7|Oct 03 2014|15:01:47|715034|||||IP = 100.100.100.2, Constructing IOS keep alive payload: proposal=32767/32767 sec.
7|Oct 03 2014|15:01:47|715076|||||Group = 100.100.100.2, IP = 100.100.100.2, Computing hash for ISAKMP
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing hash payload
7|Oct 03 2014|15:01:47|715046|||||Group = 100.100.100.2, IP = 100.100.100.2, constructing ID payload
7|Oct 03 2014|15:01:47|713906|||||IP = 100.100.100.2, Connection landed on tunnel_group 100.100.100.2
6|Oct 03 2014|15:01:47|713172|||||Group = 100.100.100.2, IP = 100.100.100.2, Automatic NAT Detection Status:     Remote end is NOT behind a NAT device     This   end is NOT behind a NAT device
7|Oct 03 2014|15:01:47|715049|||||Group = 100.100.100.2, IP = 100.100.100.2, Received DPD VID
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing VID payload
7|Oct 03 2014|15:01:47|715034|||||IP = 100.100.100.2, Processing IOS keep alive payload: proposal=32767/32767 sec.
7|Oct 03 2014|15:01:47|715076|||||Group = 100.100.100.2, IP = 100.100.100.2, Computing hash for ISAKMP
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing hash payload
7|Oct 03 2014|15:01:47|714011|||||Group = 100.100.100.2, IP = 100.100.100.2, ID_IPV4_ADDR ID received
7|Oct 03 2014|15:01:47|715047|||||Group = 100.100.100.2, IP = 100.100.100.2, processing ID payload
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) + NONE (0) total length : 96
7|Oct 03 2014|15:01:47|713236|||||IP = 100.100.100.2, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NAT-D (20) + NAT-D (20) + NONE (0) total length : 368
7|Oct 03 2014|15:01:47|713906|||||Group = 100.100.100.2, IP = 100.100.100.2, Generating keys for Responder...
7|Oct 03 2014|15:01:47|713906|||||IP = 100.100.100.2, Connection landed on tunnel_group 100.100.100.2

[Jouni Forss · Accepted solution]

Hi,

Seems to me that your NAT configuration is incorrect. You also should only need a single "nat" command to accomplish the NAT for the traffic from your source subnets to this single NATed remote subnet.

Using your configured "object" it would seem to me that you need the following "nat" configuration

nat (inside,outside) source static WIDGET-NEAR-NETWORKS WIDGET-NEAR-NETWORKS destination static WIDGET-FAR-NAT WIDGET-FAR-NETWORKS no-proxy-arp
 

This is because below is what the different parts of the "nat" command should tell the ASA

nat (sourceint,destint) souce static <real source> <mapped source> destination static <mapped destination> <real destination>

So as you can see in your original configuration you are only using the "object" that holds the NATed remote subnet and does not actually do any translation at all and it does not reference the actual subnet located at the remote site.

Then we come to the next thing. After doing this configuration you will have to notice that the Crypto ACL configured in the command "crypto map <map name> <number> match address <ACL name>" for the L2L VPN will have to match the MAPPED SOURCE and the REAL DESTINATION. This is because all NAT is performed before the L2L VPN configurations are matched. In your situation this naturally does not affect much the "source" section as you are not doing NAT for the source but you will need to use the actual real remote subnet in the Crypto ACL configurations for the packet to match the L2L VPN configurations.

This is atleast what the problem seems to me.

Hope this helps :)

- Jouni

[kpruett]

Well that makes perfect sense. I'm still stuck in looking at everything pre 8.3 NAT view, so I can see why that did not dawn on me as part of the config. I'll take a look and change, then reattempt.

Thanks for the very quick reply, Jouni!