GRE Tunnel 1841 issue; Tunnel not properly working after remote end goes down

[jonatstr]

Diagram:

                     Remote End                                                             Head End

mar 3950 -> asa 5505 vpn hard client --- wan cloud --- asa 5505 vpn server ---- 1841 head end

Gre tunnel between 3950 and 1841 head end.

EIGRP for routes back to remote end

ASA Version - Version 8.2(3)

1841 Version - C1841-ADVENTERPRISEK9-M), Version 12.4(22)YB

So this solution work great, routes are properly distributed from head end to the remote, thus client has access to all allowed resources sitting on head end.

Issue here is that is when the connection between the remote and headend goes down (  (vpn client goes down, or move to anothe subnet as the asa 5505 and mar are being used to provide a hardware contivity back to "home", portable hardware solution), there are problems with all GRE encapsulated packets from the 1841 to asa 5505

The problem I have seen is the GRE packets are not longer being encapsulated on the asa 5505 vpn server and tunnel over.  All other packets (IGMP of known routes, ectt.) get tunneled to the remote end find.  GRE encapsulated packets are being sent from the 1841 head end but not being encapsulated at the asa 5505 vpn server and tunneled to the remote end.  They are just being sent straight to the next hop router.

Observing the ASA, it is clear based on the packet capture and 'show crypto ipsec sa' encap count, the ASA is not encapsulating the tunnel packets.  The fix to the problem is to actually not on the ASA thought, it is to reboot the 1841 head end, and everything works again.  I can not find anything in the ASA vpn server or 1841 head end that may be causing the problem.  They only thing which really shouldn't matter as the ip access list picks up the encapsulated gre packet, is to add gre to the vpn list.

Any hints into possible config problems would be most appreciated.  In the mean time I'm going to try a new head end router, see what happens....

[Peter Paluch]

Hi Jonathan,

Have you been able to solve this problem? I would be very interested in learning what was the cause.

Is it perhaps possible to put a sniffer between the 1841 headend and the ASA box? It would be very interesting to capture the GRE traffic and compare the packets for any differences, no matter how insignificant they may seem. The fact that the reboot of the 1841 helps is quite surprising, however, there can be two reasons for that: either there is really something fishy going in the 1841 (you have a rather uncommon IOS version with possible bugs - I would suggest migrating to a 12.4 mainline or 12.4T if possible) or the ASA has some problems that are remedied by the fact that when 1841 restarts, it flaps its interfaces which also causes the flap of the interface on the ASA to which the 1841 is connected.

Best regards,

Peter

[jonatstr]

Yes, I did put one between the ASA and 1841, the GRE packets looked sane.  Only difference is was being sent from ASA to nexthop.  When the issue was happening, noted that the ASA did not encapsulate and send across the tunnel.  I sent the GRE packet straight to the next hop.

Tried couple things, like sysopt connection reclassify-vpn, but according to documentation which is correct,

this command only applies for LAN-to-LAN and dynamic VPNs.

Finally opened a bug for investigation a couple days ago.  Appears to be a possible bug (if you have the proper permission to bugs CSCti65270 and CSCth28251).  Appears some stateful udp flow information is not being torn down on the ASA 55xx.    "clear conn all" temp fix in the mean time.