Traffic is not passing on plain IPSec tunnel between two 892s

[irakli_n]

Have a weird case and looking for some suggestions/thougs where to dig, because I run out of the opetions.

Note: I've replaced real networkID to the one mentined below.

Topology: classical IPSec VPN tunnel between two Cisco 892s, with pre-shared key and no GRE. One 892 (branch_892) has access to the Internet via PPPoE and has three networks/vlans behind it. One VLAN is NATed to access internet via the PPPoE. Access to two other VLANs - VL92 (100.100.200.0/24) and VL93 (100.100.100.0/24) need is done thrue the VPN tunnel.

Second 892 (892_DC) has just one interface - WAN on Gigabit enabled/connected and has a static route to the default GW. It does not have any interal network defined. So the router is strictly used to send traffic for VL92/VL93 to the branch 892 via IPSec tunnel.

Here is the problem: access to/from VL93 (100.100.100.0/24) works, however for VL92 (100.100.100.0/24) - does not.

From devices in VL92 I can ping the 892_DC IP address across the VPN tunnel. From the 892_DC router I can also ping devices in VL92. However I can no ping from VL92 any device beyond the 892_DC and at the same time packet arriving on 892_DC for VL92 are not sent out via the VPN tunnel.

I took the packet trace on 892_DC using capture point/buffer to capute packets for VL92 and could see that traffic does arrive at the 892_DC. I run the same capute on Branch_892 and there was not a single packet.

So... what is wrong? More interesting I modified the access list such a way that left on VL92 and still - no packets are sent out thru the tunnel.

Any idea? Config of both routers are below

-------

892_DC#show ru

!

crypto isakmp policy 10

encr aes 256

hash sha256

authentication pre-share

group 2

crypto isakmp key *********** address 1.2.3.4

crypto isakmp keepalive 10 periodic

!

crypto isakmp peer address 1.2.3.4

description to-COIL-892

!

!

crypto ipsec transform-set IT-IPSec-Transform-Set esp-aes 256 esp-sha256-hmac

crypto ipsec df-bit clear

!

crypto map IT-IPSec-Crypto-Map 10 ipsec-isakmp

set peer 1.2.3.4

set security-association lifetime kilobytes disable

set security-association lifetime seconds 86400

set transform-set IT-IPSec-Transform-Set

match a ddress 101

reverse-route

qos pre-classify

!

interface GigabitEthernet0

ip address 10.20.30.40 255.255.255.240

ip mtu 1400

ip tcp adjust-mss 1360

duplex auto

speed auto

crypto map IT-IPSec-Crypto-Map

!

ip route 0.0.0.0 0.0.0.0 10.20.30.41

!

access-list 101 permit ip any 100.100.100.0 0.0.0.255 log

access-list 101 permit ip any 100.100.200.0 0.0.0.255 log

-------------------------------------------------------------------------------------

Branch_892#sh run

!

crypto isakmp policy 10

encr aes 256

hash sha256

authentication pre-share

group 2

crypto isakmp key ******** address 10.20.30.40

crypto isakmp keepalive 10 periodic

!

crypto isakmp peer address 10.20.30.40

!

!

crypto ipsec transform-set IT-IPSec-Transform-Set esp-aes 256 esp-sha256-hmac

crypto ipsec df-bit clear

!

crypto map IT-IPSec-Crypto-Map 10 ipsec-isakmp

set peer 10.20.30.40

set security-association lifetime kilobytes disable

set security-association lifetime seconds 86400

set transform-set IT-IPSec-Transform-Set

match address 101

reverse-route

qos pre-classify

!

interface FastEthernet6

description VL92

switchport access vlan 92

!

interface FastEthernet7

description VL93

switchport access vlan 93

!

interface GigabitEthernet0

description ## To WAN ##

no ip address

duplex auto

speed auto

pppoe-client dial-pool-number 1

!

interface Vlan1

description ## To local ##

ip address 192.168.1.254 255.255.255.0

ip nat inside

ip virtual-reassembly in

!

interface Vlan92

description fa6-nexus e100/0/40

ip address 100.100.200.1 255.255.255.0

!

interface Vlan93

description fa7-nexus e100/0/38

ip address 100.100.100.1 255.255.255.0

!

interface Dialer0

no ip address

no cdp enable

!

interface Dialer1

ip address 1.2.3.4 255.255.255.248

ip mtu 1454

ip nat outside

ip virtual-reassembly in max-reassemblies 256

encapsulation ppp

ip tcp adjust-mss 1414

dialer pool 1

dialer-group 1

ppp authentication chap callin

ppp chap hostname ~~~~~~~~~~~~~~~~~~~~

ppp chap password =====================

no cdp enable

crypto map IT-IPSec-Crypto-Map

!

dialer-list 1 protocol ip permit

!

access-list 101 permit ip 100.100.100.0 0.0.0.255 any

access-list 101 permit ip 100.100.200.0 0.0.0.255 any

!

ip route 0.0.0.0 0.0.0.0 Dialer1

[andrew.prince]

Check you are not "re-natting" the traffic, as if you are natting, it will not match the "interesting ACL" you have defined. 

Remember, NAT->Route->Encrypt.

[irakli_n]

On branch_892 there are following NAT related statements:

ip nat inside source list NATACL interface Dialer1 overload

ip nat inside source static 192.168.1.142 1.2.3.172

ip nat inside source static 192.168.1.141 1.2.3.173

ip nat inside source static 192.168.1.147 1.2.3.174

ip route 0.0.0.0 0.0.0.0 Dialer1

!

ip access-list standard NATACL

permit 192.168.1.0 0.0.0.255

!

As you can see, in the NAT ACL only the the 192.168.1.0/24 network is natted, VL92 (100.100.200.0/24) and VL93 (100.100.100.0/24) - are not.

Please once again pay attention to the following fact: from the branch_892's VL92 interface (100.100.200.1) and VL93 interface (100.100.100.1) I can ping the remote router (892_DC) IP address 10.20.30.40 across the VPN tunnel.

However I can not ping anything beyond the VPN tunnel from VL92 (from VL93 it works) and also - packets arriving at 892_DC for the VL92 network are not sent out thu the VPN tunnel: I've put wireshark on one server in VL92 and do not see any packet arriving there from the VPN tunnel.

If there would be the problem with NAT pinging/accessing would not work for both VL92 and VL93, right?

[QuakeDemon · Accepted solution]

Yes that sounds correct - so another possible issue is routing, is the routing 100% correct on both sides? Can you post both sides config for review?

[andrew.prince]

I would also say the the access-lists on both sides that define interesting should should be re-written. 

Branch_892

~Remove~

access-list 101 permit ip 100.100.100.0 0.0.0.255 any

access-list 101 permit ip 100.100.200.0 0.0.0.255 any

~ADD~

access-list 101 permit ip 100.100.100.0 0.0.0.255 <>

access-list 101 permit ip 100.100.200.0 0.0.0.255 <>

892_DC

~Remove~

access-list 101 permit ip any 100.100.100.0 0.0.0.255 log

access-list 101 permit ip any 100.100.200.0 0.0.0.255 log

~ADD~

!

access-list 101 permit ip 100.100.100.0 0.0.0.255 log

access-list 101 permit ip 100.100.200.0 0.0.0.255 log

[irakli_n]

The thing is that there is no 'remote subnet' - ALL traffic from/to above mentioned two networks needs to be encrypted and sent out via the IPSec tunnel.

But, the issue has being resolved - the problem was indeed with the routing. and particulary in the statment in bold below:

crypto map IT-IPSec-Crypto-Map 10 ipsec-isakmp

set peer 10.20.30.40

set security-association lifetime kilobytes disable

set security-association lifetime seconds 86400

set transform-set IT-IPSec-Transform-Set

match address 101

reverse-route

qos pre-classify

The problem was that there are two networks which needed to be encrypted and sent out. Having reverse-route command w/o any additional parameters works fine when there is just one network but with two 892 on other side got confused. Tunnel was flapping when IPSec SAs were cleaned at 892_DC router: so at a momemnt VL93 traffic was sent out via the tunnel and VL92 was not accessible. After cleaning IPSec SAs the opposite has happened. One more cleaning IPSec SAs and situation switched again.

This phenomenom was manifesting itself when lifetime of IPSec SAs got expired too. The interesting thing was that there was no indication whatsoever in crypto debug messages what was the problem. Only after starting debugging routing and connecting dots the picture become clearer.

To fix the problem permanenty on the the branch 892 router (the one which is attached to VL92 and VL93) command reverse-route static needs to be entered in the crypto map config.

After that everything started working as expected.