VPN Clients cannot access any internal address

Answered Question
Jul 24th, 2008

Definitely in need of some expert help on this one...

Attempting to set up VPN client access on an ASA 5520 that has been used only as a

firewall until now. The ASA was recently updated to Version 7.2(4).

Problem: Once connected, the VPN client cannot access anything. VPN client cannot

ping any address on internal networks, or even the inside interface of the ASA.

(hopefully) Relevant Details:

1) The tunnel appears to be up. The clients are local authenticated by the ASA and

are able to connect.

2) Per many other related posts, I ran a "sh crypto ipsec sa" to see the output: it

appears that packets are being decapsulated and decrypted, but NOT encapsulated or

encrypted (see output of "sh crypto ipsec sa" attached).

3) Per other related posts, we have added commands related to NAT reversal (crypto

isakmp nat-traversal 20

crypto isakmp ipsec-over-tcp port 10000). These were in fact missing from our

configuration.

4) We have tried both TCP encapsulation and UDP encapsulation with experimental client

profiles: same result in both cases.

5) If I (attempt) ping to an internal IP address from the connected client, the

realtime ASA log entries show the setup and teardown of the ICMP requests from the

client to the internal target.

6) Packet capture on the internal address (the one we are attempting to ping from the

VPN client) shows that the ICMP request was received and answered. (See attached

capture).

7) Our objective is to create about 10 different VPN client profiles, each with

different combinations of access to Internal VLANs or DMZ VLANs. We have no

preferences for encryption type or method so long as it is secure and it works: That

said, feel free to recommend a different approach entirely.

We've tried everything we can think of, so any help and/or advice would be greatly

Sanitized configuration of ASA is also attached.

appreciated!!

Thanks!

I have this problem too.
0 votes
Correct Answer by a.alekseev about 8 years 4 months ago

it should be the last step :)

on 6509

ip route 172.16.100.0 255.255.255.0 172.16.20.2

and on ASA

no route inside 172.16.40.0 255.255.255.0 172.16.20.2

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (4 ratings)
Loading.
a.alekseev Thu, 07/24/2008 - 13:01

could we simplify your confiruration?

no crypto dynamic-map outside_dyn_map 20 match address outside_cryptomap_dyn_20

no crypto dynamic-map outside_dyn_map 20 set transform-set ESP-3DES-SHA

no crypto dynamic-map outside_dyn_map 40 set pfs group5

no crypto dynamic-map outside_dyn_map 40 set transform-set ESP-3DES-SHA

no crypto dynamic-map outside_dyn_map 60 set pfs group1

no crypto dynamic-map outside_dyn_map 60 set transform-set TRANS_ESP_3DES_SHA

crypto dynamic-map outside_dyn_map 80 set transform-set ESP-3DES-SHA

goodwinscottns Thu, 07/24/2008 - 13:46

Done.

The remaining crypto-related entries are these:

crypto ipsec transform-set TRANS_ESP_3DES_SHA esp-3des esp-sha-hmac

crypto ipsec transform-set TRANS_ESP_3DES_SHA mode transport

crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

crypto dynamic-map outside_dyn_map 80 set transform-set ESP-3DES-SHA

crypto map outside_map 65535 ipsec-isakmp dynamic outside_dyn_map

crypto map outside_map interface outside

crypto isakmp identity hostname

crypto isakmp enable outside

crypto isakmp policy 10

authentication pre-share

encryption 3des

hash sha

group 2

lifetime 86400

crypto isakmp nat-traversal 20

crypto isakmp ipsec-over-tcp port 10000

Updated show crypto output is attached.

No change in behavior at this point.

Thanks for helping! Let me know what to try next...

a.alekseev Thu, 07/24/2008 - 14:21

could you try IPSec over UDP?

also

try to change

"crypto isakmp identity address"

How do you check the connection?

could you ping from the client this address 172.16.20.2

do you have a route for 172.16.100.0/24 on 172.16.20.2?

goodwinscottns Thu, 07/24/2008 - 15:49

1) Created duplicate client profile with change to use UDP, not TCP. Profile is able to establish tunnel but still cannot receive ping replies back from Lan-side server. Packet capture on server still indicates it is receiving and responding to ICMP echo requests from client on 172.16.100.11

2) Made change to "crypto isakmp identity address". Did this before test above.

3) Method for checking connection is this: VPN client software installed on PC at remote site (client version is 5.0.03.0530). Establish tunnel connection from VPN client to ASA on 111.222.167.2 (Outside interface of ASA). Ping valid machine IP addresses on LAN-side VLANS to check for response. Have also tried remote desktop etc... Use Wireshark packet capture on target machine to look for packets from VPN client machine address.

4) "do you have a route for 172.16.100.0/24 on 172.16.20.2?" I'm not sure what you mean exactly. 172.16.20.2 is the "Inside" interface of the ASA. Do you mean is there a route statement in the ASA or the switch to which 172.16.20.2 is connected? 172.16.20.2 is physically connected to a 6509 switch. The config for this interface on the 6509 along with a "show ip route" output below:

interface GigabitEthernet5/2

description Connection to ASA5500 port 1

switchport

switchport access vlan 20

switchport mode access

no ip address

media-type rj45

spanning-tree portfast

Show IP route from 6509:

Gateway of last resort is 172.16.20.2 to network 0.0.0.0

172.16.0.0/16 is variably subnetted, 23 subnets, 2 masks

C 172.16.50.0/24 is directly connected, Vlan50

C 172.16.51.0/24 is directly connected, Vlan51

C 172.16.40.0/24 is directly connected, Vlan40

C 172.16.41.0/24 is directly connected, Vlan41

C 172.16.39.0/24 is directly connected, Vlan39

C 172.16.28.0/24 is directly connected, Vlan28

C 172.16.24.0/24 is directly connected, Vlan24

C 172.16.25.0/24 is directly connected, Vlan25

C 172.16.20.0/24 is directly connected, Vlan20

C 172.16.21.0/24 is directly connected, Vlan21

C 172.16.22.0/24 is directly connected, Vlan22

C 172.16.23.0/24 is directly connected, Vlan23

C 172.16.16.0/24 is directly connected, Vlan16

C 172.16.17.0/24 is directly connected, Vlan17

C 172.16.18.0/24 is directly connected, Vlan18

C 172.16.12.0/24 is directly connected, Vlan12

C 172.16.13.0/24 is directly connected, Vlan13

C 172.16.14.0/24 is directly connected, Vlan14

C 172.16.15.0/24 is directly connected, Vlan15

C 172.16.10.0/24 is directly connected, Vlan10

C 172.16.11.0/24 is directly connected, Vlan11

D 172.16.0.0/16 is a summary, 1w1d, Null0

C 172.16.102.0/24 is directly connected, Vlan102

172.18.0.0/16 is variably subnetted, 5 subnets, 3 masks

D 172.18.20.0/24 [90/4227072] via 172.18.5.2, 6d23h, GigabitEthernet2/14

D 172.18.10.0/24 [90/4226816] via 172.18.5.2, 6d23h, GigabitEthernet2/14

D 172.18.10.1/32 [90/4226816] via 172.18.5.2, 6d23h, GigabitEthernet2/14

C 172.18.5.0/24 is directly connected, GigabitEthernet2/14

D 172.18.0.0/16 is a summary, 6d11h, Null0

D 172.21.0.0/16 [90/4227328] via 172.18.5.2, 6d23h, GigabitEthernet2/14

C 192.168.102.0/24 is directly connected, GigabitEthernet2/48

S* 0.0.0.0/0 [1/0] via 172.16.20.2

a.alekseev Thu, 07/24/2008 - 22:05

this is the answer :))

interface GigabitEthernet0/1

speed 1000

duplex full

nameif inside

security-level 100

ip address 172.16.20.2 255.255.255.0

!

interface GigabitEthernet0/2

speed 1000

duplex full

nameif dmz

security-level 50

ip address 172.16.10.2 255.255.255.0

!

route inside 172.16.50.0 255.255.255.0 172.16.20.2 1

route inside 172.16.25.0 255.255.255.0 172.16.20.2 1

route inside 172.16.24.0 255.255.255.0 172.16.20.2 1

route inside 172.16.23.0 255.255.255.0 172.16.20.2 1

route inside 172.16.22.0 255.255.255.0 172.16.20.2 1

route inside 172.16.21.0 255.255.255.0 172.16.20.2 1

route inside 172.16.51.0 255.255.255.0 172.16.20.2 1

route inside 172.16.28.0 255.255.255.0 172.16.20.2 1

route inside 172.16.39.0 255.255.255.0 172.16.20.2 1

route inside 192.168.102.0 255.255.255.0 192.168.102.250 1

route inside 192.168.100.0 255.255.255.0 192.168.102.250 1

route inside 172.18.5.0 255.255.255.0 172.16.20.2 1

route inside 172.18.10.0 255.255.255.0 172.16.20.2 1

route inside 172.18.20.0 255.255.255.0 172.16.20.2 1

route inside 172.21.100.0 255.255.255.0 172.16.20.2 1

route dmz 172.16.17.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.16.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.14.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.13.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.12.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.11.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.15.0 255.255.255.0 172.16.10.2 1

route dmz 172.16.18.0 255.255.255.0 172.16.10.2 1

ASA cannot be a default gateway for itself.

Do corrections :))

goodwinscottns Fri, 07/25/2008 - 08:22

I see your point, but all of the above route statements are already in the running config.

Is it possible we need one or more specific route statement in our 6509 switch for the VPN IP pools? Below are all statements related to routing from our 6509:

!

router eigrp 10

network 172.16.0.0

network 172.18.0.0

network 172.21.0.0

network 192.168.0.0

auto-summary

!

ip default-gateway 172.16.20.2

ip classless

ip route 0.0.0.0 0.0.0.0 172.16.20.2

!

access-list 111 permit ip 172.16.15.0 0.0.0.255 any

access-list 111 deny ip 172.16.15.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 deny ip 172.16.11.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.11.0 0.0.0.255 any

access-list 111 deny ip 172.16.12.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.12.0 0.0.0.255 any

access-list 111 deny ip 172.16.13.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.13.0 0.0.0.255 any

access-list 111 deny ip 172.16.14.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.14.0 0.0.0.255 any

access-list 111 deny ip 172.16.16.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.16.0 0.0.0.255 any

access-list 111 deny ip 172.16.17.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.17.0 0.0.0.255 any

access-list 111 deny ip 172.16.18.0 0.0.0.255 172.16.0.0 0.0.255.255

access-list 111 permit ip 172.16.18.0 0.0.0.255 any

!

route-map to_DMZ_ASA permit 10

match ip address 111

set ip next-hop 172.16.10.2

!

Speaking of routing, we're considering upgrading the ASA to version 8.x to be able to use EIGRP, which we use on every other device on our network. Do you think this would simplify our VPN routing?

Many thanks for your help with this.

a.alekseev Fri, 07/25/2008 - 08:28

this routes for your ASA

route inside 172.16.50.0 255.255.255.0 172.16.20.2 1

route inside 172.16.25.0 255.255.255.0 172.16.20.2 1

route inside 172.16.24.0 255.255.255.0 172.16.20.2 1

route inside 172.16.23.0 255.255.255.0 172.16.20.2 1

route inside 172.16.22.0 255.255.255.0 172.16.20.2 1

route inside 172.16.21.0 255.255.255.0 172.16.20.2 1

route inside 172.16.51.0 255.255.255.0 172.16.20.2 1

route inside 172.16.28.0 255.255.255.0 172.16.20.2 1

route inside 172.16.39.0 255.255.255.0 172.16.20.2 1

route inside 192.168.102.0 255.255.255.0 192.168.102.250 1

route inside 192.168.100.0 255.255.255.0 192.168.102.250 1

route inside 172.18.5.0 255.255.255.0 172.16.20.2 1

route inside 172.18.10.0 255.255.255.0 172.16.20.2 1

route inside 172.18.20.0 255.255.255.0 172.16.20.2 1

route inside 172.21.100.0 255.255.255.0 172.16.20.2 1

your asa has ip address 172.16.20.2

ASA cannot be a gatway for itself.

check all you routes for correct next-hop

I think that 172.16.20.1 belongs to 6500

sh you should change for example

route inside 172.16.50.0 255.255.255.0 172.16.20.1

goodwinscottns Fri, 07/25/2008 - 09:49

OK, I made the changes to the route inside statements so that they point to the 6509 VLAN IP addresses. For example:

Before: route inside 172.16.50.0 255.255.255.0 172.16.20.2 1

After: route inside 172.16.50.0 255.255.255.0 172.16.20.1 1

No change to original problem. Client cannot ping/access anything on the far side of the tunnel.

Back to encrypting and decrypting packets: It is still the case that we do not appear to be correctly encrypting and decrypting packets on both sides of the tunnel.

On the ASA side, we see this:

#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0

#pkts decaps: 87, #pkts decrypt: 87, #pkts verify: 87

On the VPN client side we see this:

Packets

Encrypted: 124

Decrypted: 0

Discarded: 2

Bypassed: 4760

Can you confirm that it is normal to see zero packets successfully encrypted /decrypted if in fact we have a routing problem? It seems to me that we should be seeing at least a few packets successfully encrypted /decrypted on both ends...

a.alekseev Fri, 07/25/2008 - 12:53

interface GigabitEthernet0/0

speed 1000

duplex full

nameif outside

security-level 0

ip address 111.222.167.2 255.255.255.0

route outside 0.0.0.0 0.0.0.0 111.222.167.2 1

please correct this route also...

you have routing problem

let's correct it first.

goodwinscottns Fri, 07/25/2008 - 14:52

OK, routing changes made. Route statements from current ASA configuration below.

Questions:

1) The VPN IP pool we're testing with is 172.16.100.10 - 172.16.100.254. Should there be a route statement in the ASA to route 172.16.100.0 255.255.255.0 172.16.20.1 ? This route statement IS in the current configuration.

2) Is it necessary to have a VLAN on the 6509 switch to match the IP pool for VPN clients? Currently we have VLAN 100 = 172.16.100.1 255.255.255.0 on the 6509.

Current route statements from ASA:

global (outside) 1 interface

nat (inside) 0 access-list inside_nat0_outbound

nat (inside) 1 0.0.0.0 0.0.0.0

nat (dmz) 0 access-list dmz_nat0_outbound

nat (dmz) 1 0.0.0.0 0.0.0.0

nat (Management2) 0 access-list Management2_nat0_outbound

static (dmz,outside) 111.222.167.15 172.16.15.10 netmask 255.255.255.255

static (dmz,outside) 111.222.167.33 172.16.15.33 netmask 255.255.255.255

static (dmz,outside) 111.222.167.13 172.16.13.10 netmask 255.255.255.255

static (dmz,outside) 111.222.167.12 172.16.12.11 netmask 255.255.255.255

static (dmz,outside) 111.222.167.22 172.16.12.10 netmask 255.255.255.255

static (dmz,outside) 111.222.167.23 172.16.12.12 netmask 255.255.255.255

static (dmz,outside) 111.222.167.16 172.16.16.10 netmask 255.255.255.255

static (dmz,outside) 111.222.167.6 172.16.16.6 netmask 255.255.255.255

static (dmz,outside) 111.222.167.17 172.16.17.10 netmask 255.255.255.255

static (dmz,outside) 111.222.167.18 172.16.18.18 netmask 255.255.255.255

static (dmz,outside) 111.222.167.19 172.16.18.19 netmask 255.255.255.255

static (dmz,outside) 111.222.167.20 172.16.16.12 netmask 255.255.255.255

static (inside,outside) 111.222.167.5 172.16.22.11 netmask 255.255.255.255

access-group outside_in in interface outside

route outside 0.0.0.0 0.0.0.0 111.222.167.1 1

route inside 172.16.40.0 255.255.255.0 172.16.20.1 1

route inside 172.21.100.0 255.255.255.0 172.16.20.1 1

route inside 172.18.20.0 255.255.255.0 172.16.20.1 1

route inside 172.16.100.0 255.255.255.0 172.16.100.1 1

route inside 172.16.21.0 255.255.255.0 172.16.20.1 1

route inside 172.16.22.0 255.255.255.0 172.16.20.1 1

route inside 172.16.23.0 255.255.255.0 172.16.20.1 1

route inside 172.16.24.0 255.255.255.0 172.16.20.1 1

route inside 172.16.25.0 255.255.255.0 172.16.20.1 1

route inside 172.16.28.0 255.255.255.0 172.16.20.1 1

route inside 172.16.39.0 255.255.255.0 172.16.20.1 1

route inside 172.16.50.0 255.255.255.0 172.16.20.1 1

route inside 172.16.51.0 255.255.255.0 172.16.20.1 1

route inside 172.18.5.0 255.255.255.0 172.16.20.1 1

route inside 172.18.10.0 255.255.255.0 172.16.20.1 1

route inside 192.168.100.0 255.255.255.0 172.16.20.1 1

route inside 192.168.102.0 255.255.255.0 172.16.20.1 1

route dmz 172.16.16.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.11.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.12.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.13.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.14.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.15.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.17.0 255.255.255.0 172.16.10.1 1

route dmz 172.16.18.0 255.255.255.0 172.16.10.1 1

a.alekseev Fri, 07/25/2008 - 15:04

1. delete this route

no route inside 172.16.100.0 255.255.255.0 172.16.100.1 1

2. No, you shoudn't

goodwinscottns Fri, 07/25/2008 - 15:42

OK, done. Route deleted from ASA. VLAN 100 deleted from 6509.

What's the next step?

REALLY appreciate your time and effort on this!!

a.alekseev Fri, 07/25/2008 - 22:06

add this line to the configuration

crypto dynamic-map outside_dyn_map 80 set reverse-route

try to connect

try to ping from the client 172.16.20.1

show the ouput "sh crypto ipsec sa" "sh route" "sh arp"

a.alekseev Sat, 07/26/2008 - 02:48

add to the configuration

"icmp permit any inside"

do

"clear arp"

"clear xl"

"clear local-host"

try to ping from ASA 172.16.20.1

connect vpn client

try to ping from vpn client 172.16.20.1

show the output "sh route" "sh arp" "sh crypto ipsec sa"

from 6500

sh run int vlan 20

sh ip arp vlan 20

ping 172.16.20.1

goodwinscottns Sat, 07/26/2008 - 07:24

Done.

1) ASA can ping 172.16.20.1 from Inside, no problem.

2) VPN client cannot ping 172.16.20.1 Still showing Decrypted = 0.

Requested ASA outputs attached.

Requested 6500 outputs below:

6509#sh run int vlan 20

Building configuration...

Current configuration : 62 bytes

!

interface Vlan20

ip address 172.16.20.1 255.255.255.0

end

6509#sh ip arp vlan 20

Protocol Address Age (min) Hardware Addr Type Interface

Internet 172.16.20.1 - 0015.2c19.d000 ARPA Vlan20

Internet 172.16.20.2 19 0012.d948.f207 ARPA Vlan20

6509#ping 172.16.20.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 172.16.20.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Correct Answer
a.alekseev Sat, 07/26/2008 - 09:03

it should be the last step :)

on 6509

ip route 172.16.100.0 255.255.255.0 172.16.20.2

and on ASA

no route inside 172.16.40.0 255.255.255.0 172.16.20.2

goodwinscottns Sat, 07/26/2008 - 10:15

Aleksey, you are genius! It works!

Actually route inside 172.16.40.0 255.255.255.0 172.16.20.2 did not exist, so I'm guessing you meant "no route inside 172.16.40.0 255.255.255.0 172.16.20.1" and have removed this.

Question: Why remove the route inside 172.16.40.0 255.255.255.0 172.16.20.1 ? Is this because ASA Gi0/3 has 172.16.40.210 address? Gi0/3 is set to "management only" and is currently disabled.

a.alekseev Sat, 07/26/2008 - 11:05

Cool :)

I mean "route inside 172.16.40.0 255.255.255.0 172.16.20.2"

because according provided information on ASA you have

Result of the command: "show route"

Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP

i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area

* - candidate default, U - per-user static route, o - ODR

P - periodic downloaded static route

Gateway of last resort is 111.222.167.1 to network 0.0.0.0

S 172.16.40.0 255.255.255.0 [1/0] via 172.16.20.2, inside

[1/0] via 172.16.20.1, inside

goodwinscottns Sat, 07/26/2008 - 11:48

So we need in ASA BOTH:

route inside 172.16.40.0 255.255.255.0 172.16.20.2 and route inside 172.16.40.0 255.255.255.0 172.16.20.1 ?

a.alekseev Sat, 07/26/2008 - 12:14

you need only one

no route inside 172.16.40.0 255.255.255.0 172.16.20.2

route inside 172.16.40.0 255.255.255.0 172.16.20.1

pmccubbin Mon, 09/29/2008 - 06:51

Scott,

This is a great piece of troubleshooting and it helped me better understand the whole process.

One follow-up question:

Did you ever try turning on EIGRP?

Best,

Paul

goodwinscottns Mon, 10/06/2008 - 20:11

Yes, I'm happy to say that we have EIGRP working nicely on our ASA 5520. It's good because it virtually eliminates routing as a variable. If something's not working, it's almost always an ACL or NAT issue now...

pmccubbin Sat, 10/11/2008 - 05:51

Scott,

Thanks for the confirmation. I give all your contributions to this thread a "5" from NYC.

Best,

Paul

Actions

This Discussion