×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Wierd NAT behaviour with AnyConnect clients

Answered Question
Jul 9th, 2013
User Badges:

Hi there,


I have a problem with our AnyConnect clients not being able to access a particular resource that exists on a 3rd party VPN network.


Both AnyConnect clients & 3rd Party Site to Site VPN terminate on the Outside Interface of the ASA.


There is a NAT setup between the 3rd Party network and our ASA so that we share the 192.168.40.0/24 subnet. The first /25 is for 3rd party hosts & the second /25 is for our hosts.


We are trying to access a service on 192.168.40.10             


The NAT rule that I have in place to achieve this is


Source = VPN-Subnet Dest = 192.168.40.0/25 Service = any

XLate Source = 192.168.40.129(PAT) XLate Dest = Original  XLateService = Original  


With the NAT rule like this, the webpage DOES NOT work. We get a SYN Timeout, and when looking at the logs, the AnyConnect client source address does not get PAT'd to 192.168.40.129


BUT....


if I change the NAT rule to this....


Source = VPN-Subnet Dest = 192.168.40.0/25 Service = any

XLate Source = 192.168.40.129(PAT) XLate Dest = 192.168.40.10  XLateService = Original 


THIS WORKS! The source address does get PAT'd to 192.168.40.129.


BUT.... the problem is now, that if the AnyConnect client tries to access any other IP in 192.168.40.0/25, the destination address gets changed all the time to 192.168.40.10.


I am new to ASA 8.3, so I was wondering whether I am missing something with how NAT rules have changes since earlier versions of ASA...


Can anyone help?


Thanks


Mario De Rosa

Correct Answer by Jouni Forss about 4 years 1 month ago

Hi,


The only reason for seeing a NAT rule that is configured at the very top for not getting applied are


  • The "same-security-traffic permit intra-interface" is NOT configured, but in this case it is since we have already taken "packet-tracer" output
  • Then there is naturally the chance that the NAT rules networks simply dont match the traffic that is entering the ASA
  • Then there is naturally the change of a bug which there have been several.


If there is no clear obvious reason for the NAT rules not matching then I would suggest opening a TAC case and/or upgrading/downgrading to some other software level to determine if a software bug is the cause.


I am not sure if you have mentioned the software level you are using?


- Jouni

Correct Answer by Jouni Forss about 4 years 1 month ago

Hi,


There has been some known bugs with Twice NAT / Manual NAT configurations in the new softwares. For example some problems have been caused by using same "object-group" in multiple NAT statements and some have been caused by using "object-group" inside other "object-group".


Result could have been for example that the NAT rules arent processed in the correct order.


- Jouni

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Jouni Forss Tue, 07/09/2013 - 07:58
User Badges:
  • Super Bronze, 10000 points or more

Hi,


If you dont have a large NAT configuration, could you provide it in CLI format.


I am not sure about the setup either.


You are trying to connect from a VPN Client connection to a remote network behind a L2L VPN connection?


What is the encryption domain of the L2L VPN? What are the source and destination networks?


Can you clarify the situation a bit.


- Jouni

marioderosa2008 Tue, 07/09/2013 - 08:34
User Badges:

Thanks Jouni,


yes, there are nearly 200 NAT rules... and they have all been configured through ASDM, so they are all object groups of some sort... to print out the config would be very confusing for anyone! lol


So, to clarify...


Yes, we have an anyconnect client which gets handed an IP out of a pool of addresses from say 10.200.200.0/24


The VPN client tries to access a web server with IP address of 192.168.40.10 which is on a remote site to site VPN.


The VPN networks are...


Remote network: 192.168.40.0/25

Local network: 192.168.40.128/25


There is a NAT rule which is configured as below...


source 10.200.200.0/24 dest 192.168.40.0/25 serv ANY -> xsource 192.168.40.129 (PAT) xdest Original xserv Original


Both the Anyconnect clients & Site to Site VPNs terminate on the same ASA interface, Outside.


With the NAT rule configured above, the PAT does not happen. When looking at the logs, the source address of 10.200.200.x is maintained and not changed to 192.168.40.129.


Maybe there is a NAT rule conflict or something?


I hope that makes more sense.


Mario

Jouni Forss Tue, 07/09/2013 - 09:01
User Badges:
  • Super Bronze, 10000 points or more

Hi,


Essentially if I were to configure a NAT rule on the basis of the information given by you it would look something like this


object network VPN-POOL

subnet 10.200.200.0 255.255.255.0


object network REMOTE-LAN

subnet 192.168.40.0 255.255.255.128


object network VPN-POOL-PAT

host 192.168.40.129


nat (outside,outside) source dynamic VPN-POOL VPN-POOL-PAT destination static REMOTE-LAN REMOTE-LAN


This should work just fine for the VPN Client users towards the Remote LAN network behind the L2L VPN.


Which leads me to believe that there might be problem with the NAT ordering. Some earlier NAT rule might be using parameters/objects which match the traffic and therefore override this new NAT rule.


Naturally you could just enter the above NAT configuration by addint the order/priority number of 1 which would enter it at the very top of the NAT rules. In other words it would be the first NAT configuration to be matched. In CLI format it would mean basically adding this configuration.


nat (outside,outside) 1 source dynamic VPN-POOL VPN-POOL-PAT destination static REMOTE-LAN REMOTE-LAN


The typical way I would troubleshoot this would be to use "packet-tracer". Especially in the cases of NAT problems its a very valuable tool and instantly tells you which NAT rules are hit.


So using your given information you could do the following


  • Connect using the AnyConnect VPN Client (or if there are active VPN client connections those will do also)
  • Check the IP address assigned to the VPN Client connection
  • Use this IP address as a source IP address for a "packet-tracer" test
  • See which NAT rule is hit


You could basically use the following type of "packet-tracer" command


packet-tracer input outside tcp 10.200.200.x 12345 192.168.40.10 80


Naturally interface names, IP address (10.200.200.x) and destination ports etc might change.


Then if you could post the output of that command here we would probably know what the problem is


- Jouni

marioderosa2008 Fri, 07/12/2013 - 01:35
User Badges:

Ok, i have created a new nat rule at number 1... i have duplicated the object groups so that I am not using the same ibjects more than once...


Manual NAT Policies (Section 1)

1 (outside) to (outside) source dynamic any obj-vpn-host-kier2 destination static Obj-Subnet-KierIntegration2 Obj-Subnet-KierIntegration2

    translate_hits = 0, untranslate_hits = 0

Correct Answer
Jouni Forss Fri, 07/12/2013 - 02:04
User Badges:
  • Super Bronze, 10000 points or more

Hi,


The only reason for seeing a NAT rule that is configured at the very top for not getting applied are


  • The "same-security-traffic permit intra-interface" is NOT configured, but in this case it is since we have already taken "packet-tracer" output
  • Then there is naturally the chance that the NAT rules networks simply dont match the traffic that is entering the ASA
  • Then there is naturally the change of a bug which there have been several.


If there is no clear obvious reason for the NAT rules not matching then I would suggest opening a TAC case and/or upgrading/downgrading to some other software level to determine if a software bug is the cause.


I am not sure if you have mentioned the software level you are using?


- Jouni

marioderosa2008 Fri, 07/12/2013 - 04:21
User Badges:

Hi Jouni,


I have resolved this by following up your suggesting about using duplicate object groups in NAT not being a good thing.


essentially, I have no proof, but i think that the issue was the following...


since in the new NAT format, every rule your create does both source and destination nat. i think we had two rules which were both Outside,Outside and they were both using the same object group for the static deestination NAT.


On the new rule, as soon as I created a new object for the same subnets but with a different name, it seems to work fine now...


Thanks very much!!


Mario

Jouni Forss Fri, 07/12/2013 - 04:26
User Badges:
  • Super Bronze, 10000 points or more

Hi,


Glad to hear you got it working


- Jouni

marioderosa2008 Tue, 07/09/2013 - 10:13
User Badges:

Thanks very much!

I will try your recommendations and get back to you.

Can you quickly explain the new nay syntax?

Cheers

Mario

Sent from Cisco Technical Support iPhone App

marioderosa2008 Wed, 07/10/2013 - 04:58
User Badges:

Hi Jouni,


OK, I have done what you recommended and I can definately see a difference in a working NAT rule and a non working NAT rule, but I still dont quite understand what is happening.


I have posted two screenshots, one with a working NAT rule and a Not working NAT rule...


On the working NAT rule you can see that the source IP does get dynamically changed.

But on the not working NAT rule you can see that the source does not get changed and infact, it seems to hit an "any, any" NAT rule.


can you spot why the not working NAT rule is different from the working NAT rule?


Mario

Jouni Forss Wed, 07/10/2013 - 05:09
User Badges:
  • Super Bronze, 10000 points or more

Ah now they are there,


Well the first one seems to be showing 2 different NAT rules.


I would much rather see the full output in the CLI format if possible


You could for example run the following commands


packet-tracer input outside tcp 10.127.252.100 24654 192.168.40.10 80


Also even though I cant see the contents of the objects in the RPF check of the NAT that is not working it has source and destination interface configured as "any" which I wouldnt really suggest. I am not quite what the purpose of that NAT configuration even is.


- Jouni

marioderosa2008 Wed, 07/10/2013 - 05:13
User Badges:

the NAT rule is right at the toip of the list and as you can see, there are no hits at all...


vpngw1# sh nat

Manual NAT Policies (Section 1)

1 (outside) to (outside) source dynamic Obj-AnyConnectDC1Subnet obj-vpn-host-kier destination static Obj-Subnet-KierIntegration Obj-Subnet-KierIntegration

    translate_hits = 0, untranslate_hits = 0

marioderosa2008 Wed, 07/10/2013 - 05:16
User Badges:

you can see that the RPF check is hitting a generic rule further down the list of NAT rules... I dont know why that rule is there or what it is doing.



vpngw1# packet-tracer input outside tcp 10.127.252.100 24654 192.168.40.10 80

Phase: 1
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (any,any) source static grp-vpn-ssl-enc-domain grp-vpn-ssl-enc-domain destination static ssl_vpn_assigned_network ssl_vpn_assigned_network
Additional Information:
NAT divert to egress interface outside
Untranslate 192.168.40.10/80 to 192.168.40.10/80

Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group outside_access_in in interface outside
access-list outside_access_in extended permit ip object Obj-AnyConnectDC1Subnet object obj-host-Wearekier
Additional Information:

Phase: 3
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 4
Type: FOVER
Subtype: standby-update
Result: ALLOW
Config:
Additional Information:

Phase: 5
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:

Phase: 6
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (outside,outside) source dynamic Obj-AnyConnectDC1Subnet obj-vpn-host-kier destination static Obj-Subnet-KierIntegration Obj-Subnet-KierIntegration
Additional Information:

Phase: 7
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (any,any) source static grp-vpn-ssl-enc-domain grp-vpn-ssl-enc-domain destination static ssl_vpn_assigned_network ssl_vpn_assigned_network
Additional Information:

Phase: 8
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:

Phase: 9
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Config:
Additional Information:
New flow created with id 196710245, packet dispatched to next module

Result:
input-interface: outside
input-status: up
input-line-status: up
output-interface: outside
output-status: up
output-line-status: up
Action: allow

Correct Answer
Jouni Forss Wed, 07/10/2013 - 05:17
User Badges:
  • Super Bronze, 10000 points or more

Hi,


There has been some known bugs with Twice NAT / Manual NAT configurations in the new softwares. For example some problems have been caused by using same "object-group" in multiple NAT statements and some have been caused by using "object-group" inside other "object-group".


Result could have been for example that the NAT rules arent processed in the correct order.


- Jouni

marioderosa2008 Wed, 07/10/2013 - 05:24
User Badges:

I just dont understand why the RPF check uses a completely different NAT rule in the NOT working screenshot, and uses the correct NAT rule in the working screenshot


I will try and create object groups with different names...


its a pitty you cannot just specify the subnet rather than naming an object group all the time.

marioderosa2008 Wed, 07/10/2013 - 06:29
User Badges:

Hi,


I have tried renaming all groups in this new rule, but it still does not work...


the new rule is rule number 1, so it shouldnt even be reaching rule 10 during NAT. The packet is not matching rule 1 when it should be.


Any other ideas?


thanks


Mario

marioderosa2008 Wed, 07/10/2013 - 05:11
User Badges:

I do not understand why there are two NAT rules being hit every time... is the ASA doing an RPF check or something?

Jouni Forss Wed, 07/10/2013 - 05:13
User Badges:
  • Super Bronze, 10000 points or more

Hi,


The ASA check that the same NAT rule is matched on the return direction also.


Usually though this results in a failure if they dont match. But in this case it doesnt seem to.


- Jouni

Jouni Forss Wed, 07/10/2013 - 05:04
User Badges:
  • Super Bronze, 10000 points or more

Hi,


I cant see any screenshots


- Jouni

Actions

This Discussion