08-10-2012 09:03 AM - edited 02-21-2020 06:15 PM
I recently made the possibly profound error of purchasing two ASA5505-BUN-K9s from a reseller on Amazon. I have over 23 years of bit-level experience with Cisco dating back to IGS & AGS routers, but the ASA is relatively new to me--and my situation has me wondering if all that experience with Cisco routers, switches, and PIX boxes was just a happy dream.
First, both boxes have the same license: base, VPN-DES & VPN-3DES-AES enabled, ASA 8.4(4), ASDM 6.4(9), routed mode.
Starting with the default configuration, I built VPNs using the ASDM site-to-site VPN wizard, and via CLI using a variety of Cisco and non-Cisco sources. Between attempts, I issued conf term / conf factory [address] [mask] to keep things consistent.
My problem is not so much that the VPN doesn't work as that it shows no evidence of existence other than the lines in the configuration file. I am not including the configuration because I've tried about 10 different examples and I suspect my problem is more fundamental than the specifics of the configuration.
Specifically:
One suspicion I have is that the product I received is not actually enabled for IPsec VPN even though the ASDM license key output indicates that it _is_. In any event, I've spent days banging my head against a wall and thought I'd bang my head against this forum and see if it's any easier on my brain.
08-10-2012 08:29 PM
Andrew,
In order to help relieve you of your suffering (believe me I have been in your situation) your best bet to an efficient solution is to post the configurations of both ASAs in order to see what could be the issue. You can always do a find and replace in order to keep your configuration protected.
Thanks,
Tarik Admani
*Please rate helpful posts*
08-15-2012 08:07 AM
I've posted the configurations.
Since I last posted, I've done the following:
The renumbering changes were made at the suggestion of one of a vendor's engineer who told me that my use of 16-bit prefixes in the 10 network was "unusual." If that's unusual it's time for me to retire--but I did it anyway.
Why does the ASA want to talk to the other protected network on the inside interface when there is a route pointing outside? Has the operation of "ping inside" changed since the Cisco VPN troubleshooting guide which promotes its use to bring up an IPsec tunnel?
I'm about ready to slag these ASAs, or RMA them. I'm looking forward to the day I can look back on this and laugh but I'm not laughing now.
08-11-2012 06:23 PM
As Tarik said, it's hard to give constructive input without seeing the configuration.
You ARE introducing "interesting traffic" that matches the cryptomaps called by the access-lists applied to both ASAs outside interfaces aren't you?
08-12-2012 06:13 AM
Marvin & Tarik:
I'm rebuilding the configurations again and simplifying the test network (trying to simulate the entire Internet was introducing more complexity than necessary).
As far as introducing interesting traffic on the ASAs I use ping inside on both ASAs (which is listed in the Cisco documentation as a recommended way to bring up the tunnel), and also ping on the test servers on the "inside" of each endpoint.
Interestingly, I was seeing the unencrypted traffic on the test network, and getting ICMP host unreachable from one of the machines on the test network. The most obviously explanation is that the cryptomap is improperly configured.
I will post the updated configurations today or tomorrow.
Andy
08-12-2012 06:19 AM
You might also try the packet tracer tool once you've rebuilt your configs. It is a nice way of seeing what the ASA believes should be the path of a packet through the appliance, including what access-lists and encryptions actions it believes it should take based on the running configuration.
08-14-2012 10:10 AM
Marvin,
I tried the packet trace tool on IRON (10.10.0.2) attempt an ssh connection to CARBON eth0 (10.2.0.100):
There is no explicit route to 10.2/16 (or to 10.10/16 on CONNECTOR). Is this the brain-damage in my configuration? A route doesn't seem to make sense, as the gateway is on the inside interface of an as-yet-inaccessible network.
I tried the following:
IRON: route outside 10.2.0.0 255.255.0.0 10.2.0.2
CONNECTOR: route outside 10.10.0.0 255.255.0.0 10.10.0.2
This time the outside interface is selected but the packet is _dropped_ by the implicit rule. I can start messing with the rules, but my understanding is that the VPN should punch through without modifying the rule set.
My configurations have "sysopt connection permit-vpn" though it doesn't show up in the configuration dump. As I said before, however, "show run sysopt" does not produce any output.
08-12-2012 06:23 AM
Hi there,
See if the interesting traffic access list is getting any hits, if not, then thats your problem
Sent from Cisco Technical Support Android App
Sent from Cisco Technical Support Android App
08-14-2012 09:41 AM
Warren,
You appear to have called it: show access-list VPN_cryptomap_10 shows no hits after "ping inside 10.2.0.100" (on IRON) and "ping inside 10.10.0.100" (on CONNECTOR). _Why_ the access list is not getting hit is still a mystery to me. Here are the configurations, so you guys can all have a belly-laugh at my expense when you find the obvious errors.
Please note that my configuration reference for these configurations was Cisco ASA: All-In-One [...] (second edition), Safari Books Online. This reference is no 100% current--for example the section on NAT traversal uses access lists instead of "crypto isakmp nat-traversal 20" recommended by the Cisco VPN troubleshooting guide. It also uses a deprecated format of the nat configuration directive for NAT traversal. NAT would be a good explanation of why the crypto map access list is not getting hit...
Configurations were both started from "conf factory" and manual entries added for VPN based on reference listed above.
Tested with "ping inside [other_inside_address]"
The test enironment:
Since I'm submitting this over RDP and I'm lazy, I'm pasting the configurations rather than attaching them as files.
-------- Start configuration for ASA 5505 "CONNECTOR" --------
: Saved
: Written by enable_15 at 15:43:47.159 UTC Tue Aug 14 2012
!
ASA Version 8.4(4)
!
hostname connector
enable password QrsXFF/pyCwKfCOJ encrypted
passwd 2KFQnbNIdI.2KYOU encrypted
names
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
interface Vlan1
nameif inside
security-level 100
ip address 10.2.0.2 255.255.0.0
!
interface Vlan2
nameif outside
security-level 0
ip address 208.105.184.122 255.255.255.248
!
route outside 208.105.184.128 255.255.255.248 208.105.184.121
!
ftp mode passive
object network obj_any
subnet 0.0.0.0 0.0.0.0
access-list VPN_cryptomap_10 remark VPN encryption 99WDR to 1401PR
access-list VPN_cryptomap_10 extended permit ip 10.2.0.0 255.255.0.0 10.10.0.0 255.255.0.0
pager lines 24
logging asdm informational
mtu outside 1500
mtu inside 1500
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
!
object network obj_any
nat (inside,outside) dynamic interface
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
http server enable
http 10.2.0.0 255.255.0.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec ikev1 transform-set 99WDR esp-aes-256 esp-sha-hmac
crypto map VPN_cryptomap 10 match address VPN_cryptomap_10
crypto map VPN_cryptomap 10 set peer 208.105.184.130
crypto map VPN_cryptomap 10 set ikev1 transform-set 99WDR
crypto map VPN_cryptomap interface outside
crypto ikev1 enable outside
crypto ikev1 policy 1
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400
telnet timeout 5
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
dhcpd auto_config outside
!
dhcpd address 10.2.0.6-10.2.0.37 inside
dhcpd enable inside
!
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
tunnel-group 208.105.184.130 type ipsec-l2l
tunnel-group 208.105.184.130 ipsec-attributes
ikev1 pre-shared-key B@dP@33word!
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
!
service-policy global_policy global
prompt hostname context
call-home reporting anonymous prompt 2
Cryptochecksum:9b49e6c8fe9aca29ff61d1030e1fb4a7
: end
-------- End configuration for ASA 5505 "CONNECTOR" --------
-------- Begin configuration for ASA 5505 "IRON" --------
: Saved
: Written by enable_15 at 15:34:49.899 UTC Tue Aug 14 2012
!
ASA Version 8.4(4)
!
hostname iron
enable password 8Ry2YjIyt7RRXU24 encrypted
passwd 2KFQnbNIdI.2KYOU encrypted
names
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
interface Vlan1
nameif inside
security-level 100
ip address 10.10.0.2 255.255.0.0
!
interface Vlan2
nameif outside
security-level 0
ip address 208.105.184.130 255.255.255.248
!
route outside 208.105.184.120 255.255.255.248 208.105.184.129
!
ftp mode passive
object network obj_any
subnet 0.0.0.0 0.0.0.0
access-list VPN_cryptomap_10 remark VPN cryptomap for 1401PR
access-list VPN_cryptomap_10 extended permit ip 10.10.0.0 255.255.0.0 10.2.0.0 255.255.0.0
pager lines 24
logging asdm informational
mtu outside 1500
mtu inside 1500
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
!
object network obj_any
nat (inside,outside) dynamic interface
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
http server enable
http 10.10.0.0 255.255.0.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec ikev1 transform-set 1401PR esp-aes-256 esp-sha-hmac
crypto map VPN_cryptomap 10 match address VPN_cryptomap_10
crypto map VPN_cryptomap 10 set peer 208.105.184.122
crypto map VPN_cryptomap 10 set ikev1 transform-set 1401PR
crypto map VPN_cryptomap interface outside
crypto ikev1 enable outside
crypto ikev1 policy 1
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400
telnet timeout 5
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
dhcpd auto_config outside
!
dhcpd address 10.10.0.6-10.10.0.37 inside
dhcpd enable inside
!
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
tunnel-group 208.105.184.122 type ipsec-l2l
tunnel-group 208.105.184.122 ipsec-attributes
ikev1 pre-shared-key B@dP@33word!
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
!
service-policy global_policy global
prompt hostname context
Cryptochecksum:ecefe71ff5a587cbc85b2a57f29d4a35
: end
-------- End configuration for ASA 5505 "IRON" --------
08-14-2012 01:07 PM
Since my last posting, read further in Cisco VPN troubleshooting guide (dated--latest ASA version described is in the 7s) and learned that I shoould include routes to the protected network. I still say WTF (what the heck), but who am I to argue:
CONNECTOR:
route outside 0.0.0.0 0.0.0.0 208.105.184.121
route outside 10.10.0.0 255.255.0.0 208.105.184.130
IRON:
route outside 0.0.0.0 0.0.0.0 208.105.184.129
route outside 10.2.0.0 255.255.0.0 208.105.184.122
In both cases I zeroed out the old 29-bit prefix routes with "no route" commands.
Now when I run packet trace, I get "flow is denied by configured rule." This message persists regardless of how many permissive rules I apply in what direction, on what interface, or in any combination. Is this a feature??
Someone help me with the order of operations, because this is what I assume should happen:
1. Packet enters INSIDE interface
2. Route looked up--FOUND, matched to OUTSIDE interface
3. Crypto map looked up on OUTSIDE interface
4. Source/destination addresses compared against map--MATCHED
5. Initiate ISAKMP/IPSEC negotiations to bring up tunnel
6. Bypass NAT and ACLs and shoot the encrytped ESP packets to the peer
This implies to me that (4) is failing. Since my access lists are correct at both ends, that means the source and/or destination aren't matching. That implies NAT.
Onward!
08-14-2012 01:50 PM
NAT appears to be a bust. I've attached the current configurations as files (stopped being lazy, or more accurately found the advanced editor). This did _apparently_ buy me an additional step in the packet trace: ACCESS-LIST, ROUTE-LOOKUP, ACCESS-LIST, and then "flow denied by configured rule."
However, both access lists appear to be implicit rules.
Result of the command: "show access-list VPN_cryptomap_10"
access-list VPN_cryptomap_10; 1 elements; name hash: 0x8fd3c8c4
access-list VPN_cryptomap_10 line 1 remark VPN cryptomap for 1401PR
access-list VPN_cryptomap_10 line 2 extended permit ip object ARUNDEL object KENNEBUNKPORT (hitcnt=0) 0x6ede1b13
access-list VPN_cryptomap_10 line 2 extended permit ip 10.10.0.0 255.255.0.0 10.2.0.0 255.255.0.0 (hitcnt=0) 0x6ede1b13
Result of the command: "show nat"
Manual NAT Policies (Section 1)
1 (inside) to (outside) source static ARUNDEL ARUNDEL destination static KENNEBUNKPORT KENNEBUNKPORT
translate_hits = 0, untranslate_hits = 0
Auto NAT Policies (Section 2)
1 (inside) to (outside) source dynamic obj_any interface
translate_hits = 0, untranslate_hits = 0
Giving up for the day--I hope someone will have mercy and point out my errors!
08-20-2012 02:51 PM
Still looking for a solution to this issue--obtained SMARTnet agreements, upgraded software (though apparently the boxes were at most recent release), went through many rounds of testing, brought in outside ringers (Cisco experts), no one understands what is going on here.
Help me Cisco Support Community, you're my only hope.
08-23-2012 10:07 AM
I have a working though not-ready-for-production IPsec L2L VPN configuration.
Everything I did until today led me to the conclusion that it was a NAT issue and evidently I was right.
My mistake before was evidently placing too much faith in ASDM and the default (factory) configuration. The factory configuration includes a PAT rule which remains regardless of what you set in the ASDM startup wizard. Also, ASDM is either broken or uses incorrect language to describe the "bypass NAT" option in the VPN wizard (the language implies that "Bypass NAT" really means "don't bypass NAT."
The configuration isn't final because I actually need PAT in place, and the VPN traffic needs to bypass it--but now I know the basic VPN functionality works and knowing is half the battle.
The working configurations are attached.
08-23-2012 11:12 AM
In the process of experimentation I added a PAT rule to one side of the connection:
nat (inside,outside) source dynamic any interface
Which broke the VPN connection. Since NAT traversal is supposed to be on by default, this makes no sense.
On both sides I added the PAT rule back and added a rule to force protected network traffic to NAT to itself (seems superfluous, but multiple sources recommend it):
object network ARUNDEL
subnet 10.10.0.0 255.255.0.0
object network KENNEBUNKPORT
subnet 10.4.0.0 255.255.0.0
nat (inside,outside) source static KENNEBUNKPORT KENNEBUNKPORT destination static ARUNDEL ARUNDEL
Doesn't work--ASA prefers the PAT rule and bypasses the more specific rule for protected traffic.
Evidently Cisco L2L VPN does not permit any form of NAT, nor does it honor NAT traversal settings, or adhere to any of the other recommendations and requirements described in multiple Cisco and third-party documents.
08-24-2012 04:00 PM
Problem solved. It's just a matter of knowing when to trust ASDM and when to ignore ASDM. I should note that because of my test environment and my personal inclination towards CLI I wasn't able to use ASDM. For most of the work. When I rebuilt by test environment and was able to use ASDM things worked pretty quickly.
In this case I'm not entirely sure what the difference was between my CLI attempts and the ASDM configuration, but it was definitely a NAT issue.
I bitched to Cisco about the fact that ASDM's treatment of NAT exemption for VPN traffic is confusticating [sic]:
- When configuring the VPN you have the option to exempt the VPN traffic from NAT. The description is (my emphasis):
Exempt ASA side host/network from address translation.
- Before you say "duh," please note that I did this the first time I configured the ASAs, long before I started this thread--it didn't work. Subsequently I deselected the option because the wizard VPN summary say (my emphasis):
Network Address Translation: The protected traffic is subjected to network address translation.
This is technically true, because a nat statement is inserted to essentially map each address to itself--but semantically it's wrong.
Here is a working configuration.
hostname connectordomain-name nmillc.net
enable password ***** encrypted
passwd ***** encrypted
names
!
interface Ethernet0/0
switchport access vlan 2
!
interface Ethernet0/1
!
interface Vlan1
nameif inside
security-level 100
ip address 10.4.0.2 255.255.0.0
!
interface Vlan2
nameif outside
security-level 0
ip address 208.105.184.122 255.255.255.248
!
ftp mode passive
dns server-group DefaultDNS
domain-name nmillc.net
object network obj_any
subnet 0.0.0.0 0.0.0.0
object network NTL1
subnet 10.4.0.0 255.255.0.0
description Test network 1 (NTL1)
object network ONET1
subnet 10.1.0.0 255.255.0.0
description Office network 1 (ONET1)
object network ONET2
subnet 10.2.0.0 255.255.0.0
description Office network 2 (ONET2)
object network ONET3
subnet 10.10.0.0 255.255.0.0
description Office network 3 (ONET3)
object network SDMZ
subnet 10.3.0.0 255.255.0.0
description Secure DMZ (SDMZ)
object network SNET
subnet 10.0.0.0 255.255.0.0
description Server network (SNET)
object-group network KAOSdescription Data center networks
network-object object NTL1
network-object object ONET1
network-object object ONET2
network-object object SDMZ
network-object object SNET
object-group network MISCHIEF
description Remote office networks
network-object object ONET3
access-list outside_cryptomap extended permit ip object-group KAOS object-group MISCHIEF
pager lines 24
logging asdm informational
mtu outside 1500
mtu inside 1500
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
nat (inside,outside) source static KAOS KAOS destination static MISCHIEF MISCHIEF no-proxy-arp route-lookup
!
object network obj_any
nat (inside,outside) dynamic interface
!
nat (inside,outside) after-auto source dynamic any interface
route outside 0.0.0.0 0.0.0.0 208.105.184.121 1
route inside 10.0.0.0 255.255.0.0 10.4.0.1 1
route inside 10.1.0.0 255.255.0.0 10.4.0.1 1
route inside 10.2.0.0 255.255.0.0 10.4.0.1 1
route inside 10.3.0.0 255.255.0.0 10.4.0.1 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
http server enable
http 10.4.0.0 255.255.0.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec ikev1 transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmaccrypto map outside_map 1 match address outside_cryptomap
crypto map outside_map 1 set pfs
crypto map outside_map 1 set peer 69.193.101.22
crypto map outside_map 1 set ikev1 transform-set ESP-AES-256-SHA
crypto map outside_map interface outside
crypto ikev1 enable outside
crypto ikev1 policy 30
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400
telnet timeout 5
ssh timeout 5
ssh key-exchange group dh-group1-sha1
console timeout 0
dhcpd auto_config outside
!
dhcpd address 10.4.0.224-10.4.0.254 inside
dhcpd dns 10.3.1.12 interface inside
dhcpd domain nmillc.net interface inside
dhcpd enable inside
!
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
group-policy GroupPolicy_69.193.101.22 internal
group-policy GroupPolicy_69.193.101.22 attributes
vpn-tunnel-protocol ikev1
tunnel-group 69.193.101.22 type ipsec-l2l
tunnel-group 69.193.101.22 general-attributes
default-group-policy GroupPolicy_69.193.101.22
tunnel-group 69.193.101.22 ipsec-attributes
ikev1 pre-shared-key *****
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum client auto
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect ip-options
!
service-policy global_policy global
prompt hostname context
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide