Cannot reach hosts behind router via VPN with IOS 15

Unanswered Question
Jul 22nd, 2010

I've deployed several routers with this setup on 12.x but with IOS 15 that was preinstalled on a new 888 I am out of luck. I have a succesfull VPN session with the router and can connet to it through the VPN but am unable to reach any other host behind it, this despite having a correct ACL in place and the router address set correctly on the clients. Please review my config:

version 15.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname d0c
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
enable secret 5 xxx
!
aaa new-model
!
aaa authentication login d0c-vpn local
aaa authorization network d0c-vpn local
!

aaa session-id common
memory-size iomem 10
!
ip source-route
!
ip dhcp excluded-address 192.168.10.254
ip dhcp excluded-address 192.168.10.10 192.168.10.20
!
ip dhcp pool ccp-pool
   import all
   network 192.168.10.0 255.255.255.0
   default-router 192.168.10.254
   lease 0 2
!
!
ip cef
no ip domain lookup
no ipv6 cef
!
username ciscoadmin privilege 15 secret 5 xxx
username d0c password 0 xxx
!
!
controller DSL 0
mode atm
dsl-mode shdsl symmetric annex B
!
!
!        
crypto isakmp policy 2
encr 3des
hash md5
authentication pre-share
group 2
lifetime 43200
!
crypto isakmp client configuration group d0c-vpn
key xxx
pool d0c
acl 102
save-password
crypto isakmp profile VPNclient
   match identity group d0c-vpn
   client authentication list d0c-vpn
   isakmp authorization list d0c-vpn
   client configuration address respond
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set d0c-ts esp-3des esp-sha-hmac
!
crypto ipsec client ezvpn d0c-vpn-profile
connect auto
group d0c-vpn-profile key xxx
mode client
xauth userid mode interactive
!
!
crypto dynamic-map d0c-map 1
set transform-set d0c-ts
set isakmp-profile VPNclient
reverse-route
!
!
!
crypto map d0c-map isakmp authorization list d0c-vpn
crypto map d0c-map client configuration address respond
!
crypto map static-map 1 ipsec-isakmp dynamic d0c-map
!

interface BRI0
no ip address
encapsulation hdlc
shutdown
isdn termination multidrop
!
interface ATM0
no ip address
load-interval 30
no atm ilmi-keepalive
pvc KPN 2/32
  encapsulation aal5mux ppp dialer
  dialer pool-member 1
!
!
interface FastEthernet0
switchport access vlan 100
!
interface Vlan1
ip address 192.168.10.254 255.255.255.0
ip nat inside
ip virtual-reassembly
ip tcp adjust-mss 1452
crypto ipsec client ezvpn d0c-vpn-profile inside
!
interface Vlan100
ip address dhcp
ip nat outside
ip virtual-reassembly
crypto map static-map
crypto ipsec client ezvpn d0c-vpn-profile
!
interface Dialer0
ip address negotiated
ip nat outside
ip virtual-reassembly
encapsulation ppp
dialer pool 1
dialer-group 1
ppp pap sent-username xxx password 0 xxx
no cdp enable
!
ip local pool d0c 192.168.222.222 192.168.222.244
ip forward-protocol nd
no ip http server
ip http access-class 23
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000
!
ip nat inside source list 101 interface Vlan100 overload
ip route 0.0.0.0 0.0.0.0 188.xxx.xxx.65
!
access-list 23 permit 192.168.10.0 0.0.0.255
access-list 23 permit 192.168.222.0 0.0.0.255
access-list 101 deny   ip any 192.168.222.0 0.0.0.255
access-list 101 permit ip 192.168.10.0 0.0.0.255 any
access-list 102 permit ip 192.168.10.0 0.0.0.255 any
dialer-list 1 protocol ip permit
no cdp run

---

sh ip route

S*    0.0.0.0/0 [1/0] via 188.xxx.xxx.65
      62.xxx.xxx.0/32 is subnetted, 1 subnets
C        62.xxx.xxx.56 is directly connected, Dialer0
      92.xxx.xxx.0/32 is subnetted, 1 subnets
C        92.xxx.xxx.169 is directly connected, Dialer0
      188.xxx.xxx.0/16 is variably subnetted, 2 subnets, 2 masks
C        188.xxx.xxx.64/29 is directly connected, Vlan100
L        188.xxx.xxx.67/32 is directly connected, Vlan100
      192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.10.0/24 is directly connected, Vlan1
L        192.168.10.254/32 is directly connected, Vlan1
      192.168.222.0/32 is subnetted, 6 subnets
S        192.168.222.235 [1/0] via 82.xxx.xxx.235, Vlan100

---

To clarify: This router has 2 WAN connections, 1 via VLAN 100 and 1 via Dialer0. I am currently not using Dialer0 but the idea is to route the VPN sessions through Dialer0 at some point and route all other traffic via VLAN 100 but for now the goal is to get the VPN working.

I am not 100% sure this is a IOS 15 related problem but given the fact that I used a similar approach on 5 other routers it seems to be the most likely candidate.

Thanks for your help in advance.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Jennifer Halim Thu, 07/22/2010 - 13:38

The only configuration that might cause the problem is the "ip tcp adjust-mss 1452" on vlan 1 as the value is a little bit high. I would recommend that you try lowering the MSS value to 1300 as follows:

interface vlan 1

     ip tcp adjust-mss 1300

Also, can you share the output of the following so we can see where exactly it's failing:

- show cry isa sa

- show cry ipsec sa

sandervanloosbroek Thu, 07/22/2010 - 13:49

Adjusting the size didn't help, 1452 should be the exact limit for a DSL connection. In my experience a too high number will overfragment the packets but would allow simple traffic like ping, this isn't the case. Here are the dumps:

---

d0c#show cry isa sa 
IPv4 Crypto ISAKMP SA
dst             src             state          conn-id status
188.xxx.xxx.67  82.xxx.xxx.235   QM_IDLE           2016 ACTIVE

IPv6 Crypto ISAKMP SA

d0c#show cry ipsec sa

interface: Vlan100
    Crypto map tag: static-map, local addr 188.xxx.xxx.67

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (192.168.10.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (192.168.222.233/255.255.255.255/0/0)
   current_peer 188.xxx.xxx.138 port 500
     PERMIT, flags={}
    #pkts encaps: 2, #pkts encrypt: 2, #pkts digest: 2
    #pkts decaps: 9, #pkts decrypt: 9, #pkts verify: 9
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 188.xxx.xxx.67, remote crypto endpt.: 188.xxx.xxx.138
     path mtu 1500, ip mtu 1500, ip mtu idb Vlan100
     current outbound spi: 0x0(0)
     PFS (Y/N): N, DH group: none

     inbound esp sas:

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:

     outbound ah sas:

     outbound pcp sas:

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (192.168.10.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (192.168.222.235/255.255.255.255/0/0)
   current_peer 82.xxx.xxx.235 port 40109
     PERMIT, flags={}
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
    #pkts decaps: 41, #pkts decrypt: 41, #pkts verify: 41
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 188.200.159.67, remote crypto endpt.: 82.171.51.235
     path mtu 1500, ip mtu 1500, ip mtu idb Vlan100
     current outbound spi: 0x4596B(285035)
     PFS (Y/N): Y, DH group: group2

     inbound esp sas:
      spi: 0x1E7DCD61(511561057)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel UDP-Encaps, }
        conn id: 27, flow_id: Onboard VPN:27, sibling_flags 80000046, crypto map: static-map
        sa timing: remaining key lifetime (k/sec): (4421038/20631)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x4596B(285035)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel UDP-Encaps, }
        conn id: 28, flow_id: Onboard VPN:28, sibling_flags 80000046, crypto map: static-map
        sa timing: remaining key lifetime (k/sec): (4421044/20631)
        IV size: 8 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

Jennifer Halim Thu, 07/22/2010 - 13:56

Base on that information, packets get to the router, and being decrypted, however, there are no traffic that is being encrypted from the router.


There is a possibility that the internal host does not have route for 192.168.222.0/24 back towards the router internal interface of 192.168.10.254? Is the internal host default gateway configured to be the router IP of 192.168.10.254?

Another test would be to see if you can ping the router interface from the vpn client. From VPN Client, try to ping 192.168.10.254. If you can a reply that means it is not problem with the VPN tunnel itself. You might want to check on the internal host themselves.

Another possibility would be if you have personal firewall installed on the host, it might block inbound ping from different subnet, hence it's not working when you try to access it from vpn client.

Hope that helps.

sandervanloosbroek Thu, 07/22/2010 - 14:07

I *can* ping the router interface so no problem there. I tried a different VPN client to be sure but the same result. I have 3 different devices over there including an Iomega StorCenter appliance and 2 HP printers, all setup correctly and none of them responds to a ping.

I ruled out a problem with the tunnel, the most likely candidate is a problem with the internal routing and/or the ACL. Both seem to be fine however which only makes this more confusing.

Thanks for your help so far, glad I didn't just 'miss' something.

Jennifer Halim Thu, 07/22/2010 - 14:10

I would try to telnet on the port that those internal hosts are listening on, and see if you get connection prompt instead of testing with ping as sometimes those devices especially printers might not respond to ping.

Jennifer Halim Thu, 07/22/2010 - 14:18

Sounds like an MSS issue if TCP connection does not work. The value of 1452 might be a little bit too high because after the traffic is being encrypted it will add an extra ESP header which could make the value over 1500. Try lowering it to 1300, reconnect via the vpn client, and try to telnet on a specific port that those servers are listening on.

You can test to telnet from the router itself on specific port and make sure that it's working from the router first.

sandervanloosbroek Thu, 07/22/2010 - 14:22

I lowered it earlier based on your suggestion but for the tunnel it didn't matter. I tried pinging and telnetting from the router (good thinking) and that
works fine:

d0c#ping 192.168.10.10

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/8 ms
d0c#telnet 192.168.10.10 80
Trying 192.168.10.10, 80 ... Open

So it has to be a problem with routing the incoming packets from the interface but I really can't think of anything.

Jennifer Halim Thu, 07/22/2010 - 15:06

You can also try ping or telnet sourcing it from a different interface of the router, and see if it responds. If it doesn't, then definitely problem with default gateway/route on the host itself.

sandervanloosbroek Fri, 07/23/2010 - 01:13

I double-checked this but the default gateway is correctly set on the hosts. Any other suggestions?

sandervanloosbroek Fri, 07/23/2010 - 02:05

To add to the curiousity: I slammed together a quick SSL VPN and that works fine with the local hosts.

brian-schultz Tue, 12/14/2010 - 05:47

Sander,

Did you ever get resolution to this issue?  I am having the exact same problem.  I just upgraded a 2800 to 15.0(1)M4.  All of my site-to-site and ezvpn tunnels are active, but packets are not being encapsulated at the router across the IPSec tunnel.  It almost seems like a routing problem because I can ping from the router across the VPN to the remote sites just fine.

sandervanloosbroek Tue, 12/14/2010 - 06:28

Unfortunately I didn't. I had 3 different routers that all had the same problem. The EasyVPN setup simply wouldn't work. Downgrading to 12.4 solved it on 1 router but the other 2 would only work with 15.0. I'm sure they changed something but I never figured out what. In the end I upgraded the configs to SSL VPN (WebVPN) which works a lot better anyways. I wasn't getting paid for all this research after all and the customer was happy with the advantages of an SSL VPN. In your case I would downgrade if you don't have the time.

I'm still curious why I never got it to work.

brian-schultz Mon, 12/20/2010 - 07:18

This seems to be a bug in 15.0(1)Mx code.  I have the same behavior with ezvpn on M3 and M4.

It works just fine in 12.4(24)T3 and I found that it also works fine in 15.1(3)T.

I have a TAC case opened for the bug, but they don't have any documented caveats yet that they could find.

Actions

This Discussion