Solved: VPN issue, have workaround but would like to know the root cause.

richard.jackson · ‎06-30-2010

I have a customer that has two sites linked over a VPN. The VPNs are terminated on two ASA (5510 & 5505). There are 5 Subnets involved.

Subnet 1 Main site Data

Subnet 2 Main site Voice-Phones

Subnet 3 Main site Voice- Servers

Subnet 4 remote site Data

Subnet 5 remote site Voice

The issue they were experiencing was a call would be silent in both directions for exactly 1 Min 10 Sec, then voice would be fine for the about 2 – 3 hours on any handset.

Testing so far is:- call set up and waited for 1 Min 10 Sec voice stream started.

2^nd Call made and ok.

VPN dropped and re-established call made, and is silent , call terminated within 30 sec

2^nd call made 1 min later, voice in both direction stright away.

I have dropped the encryption of the VPN from AES128 to 3DES, Fault still reoccurs

Fix, I have set up a GRE tunnel from their two voice Gateways (one at main site and one in remote office). The GRE routes voice traffic only and the ASA still encrypts the traffic.

The fault has not reoccurred.

I would like anyone’s views on what they think the issue might be.

Regards

Rich

Kevin Redmon · ‎07-07-2010

Richard,

If there is a different access-list entry, that will be a different Phase-2/IPSEC tunnel. From a Phase-1/ISAKMP standpoint, these should be the same assuming these connections are to the same peer.

The bug ID that I was able to locate is CSCsm50856. You can reference the details of this bug via our Bug Toolkit:

http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs

8.0(3)6 is affected by this bug. The fix is implemented in 8.0(3)13 - upgrading beyond that release should immunize you against that bug as well as several hundred bugs that have been fixed since 8.0(3)6. Before upgrading, when you reproduce the issue, you may want to do the following:

For a "non-working" recreate:

1.) show cry ipsec sa | inc ident|peer|spi

2.) run the test

3.) show cry ipsec sa | inc ident|peer|spi

For a working recreate:

1.) show cry ipsec sa | inc ident|peer|spi

2.) run the test

3.) show cry ipsec sa | inc ident|peer|spi

If you are hitting the bug or just facing Phase-2/IPSEC issues, you'll see that the relevant access-list entry isn't established in #1 of the "non-working" instance. In the "working" condition, you'll likely see the relevant access-list entry present.

If this is indeed the issue, please be sure to let us know and mark this response as answered.

Thanks,

Kevin

View solution in original post

Kevin Redmon · ‎07-06-2010

Rich,

What version of software are the ASA's running? I vaguely recall a bug where the establishment of the tunnel met a race condition - where the efforts of one side to establish a tunnel were conflicting with the other end's attempts. First attempts to bring up the tunnel failed. The endpoints would then "back-off" and retry a short time later. With the version information from both endpoints, I'll confirm if this bug was resolved in your release.

As to why the GRE tunnel is keeping this issue from occuring, are there periodic hellos being sent over the GRE tunnel - enough to keep the IPSEC tunnel up?

I look forward to your response.

Best Regards,

Kevin

richard.jackson · ‎07-07-2010

Hi Kevin

The Version both ASA are running is asa803-6-k8.

It is an interesting idea, one thing I noticed is that the VPN was always up and running the SCCP keepalives between the CCM and remote phones ensured that there was always traffic. I do not know the in-depth process that the VPN uses, but there would have been constant traffic between 172.16.18.0/24 (remote phones) to 172.16.8.0/24 (CCM Server Subnet); however when calls happen the IP flow would have been between 172.16.18.0/24 to 172.16.9.0/24, would this new subnet cause the ASA to re-do its algorithms and as such the proposed bug causing the issue?

Kevin Redmon · ‎07-07-2010

Richard,

If there is a different access-list entry, that will be a different Phase-2/IPSEC tunnel. From a Phase-1/ISAKMP standpoint, these should be the same assuming these connections are to the same peer.

The bug ID that I was able to locate is CSCsm50856. You can reference the details of this bug via our Bug Toolkit:

http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs

8.0(3)6 is affected by this bug. The fix is implemented in 8.0(3)13 - upgrading beyond that release should immunize you against that bug as well as several hundred bugs that have been fixed since 8.0(3)6. Before upgrading, when you reproduce the issue, you may want to do the following:

For a "non-working" recreate:

1.) show cry ipsec sa | inc ident|peer|spi

2.) run the test

3.) show cry ipsec sa | inc ident|peer|spi

For a working recreate:

1.) show cry ipsec sa | inc ident|peer|spi

2.) run the test

3.) show cry ipsec sa | inc ident|peer|spi

If you are hitting the bug or just facing Phase-2/IPSEC issues, you'll see that the relevant access-list entry isn't established in #1 of the "non-working" instance. In the "working" condition, you'll likely see the relevant access-list entry present.

If this is indeed the issue, please be sure to let us know and mark this response as answered.

Thanks,

Kevin

richard.jackson · ‎07-08-2010

Hi Kevin

The customer does not want to move from the GRE tunnel, however I am going to Lab this and let you know, however I have rated this response many thanks for your help.

rich