ipSEC vpn dropped every day in peak hours - Page 2

DTSCanada · ‎06-23-2010

Hi, there,

our l2l vpn keeps dropping around 11am and 4pm, never happened during night and weekend.

the main errior is"Removing peer from correlator table failed, no match!"

the detail log is attached(from asa side)

pls help me with this

we are using asa5510 7.0 and pix 501 6.3

part of the log

"4|Jun 23 2010 10:31:00|713903: IP = x.x.x.138, Error: Unable to remove PeerTblEntry
3|Jun 23 2010 10:31:00|713902: IP = x.x.x.138, Removing peer from peer table failed, no match!
7|Jun 23 2010 10:31:00|713906: IP = x.x.x.138, sending delete/delete with reason message
7|Jun 23 2010 10:31:00|713906: IP = x.x.x.138, IKE SA MM:4d24e98b terminating: flags 0x01000002, refcnt 0, tuncnt 0
7|Jun 23 2010 10:31:00|715065: IP = x.x.x.138, IKE MM Responder FSM error history (struct &0x1930e90) <state>, <event>: MM_DONE, EV_ERROR-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent-->MM_SND_MSG2, EV_SND_MSG-->MM_SND_MSG2, EV_START_TMR-->MM_SND_MSG2, EV_RESEND_MSG-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent
4|Jun 23 2010 10:30:54|713903: IP = x.x.x.138, Error: Unable to remove PeerTblEntry
3|Jun 23 2010 10:30:54|713902: IP = x.x.x.138, Removing peer from peer table failed, no match!
7|Jun 23 2010 10:30:54|713906: IP = x.x.x.138, sending delete/delete with reason message
7|Jun 23 2010 10:30:54|713906: IP = x.x.x.138, IKE SA MM:57d21f6d terminating: flags 0x01000002, refcnt 0, tuncnt 0
7|Jun 23 2010 10:30:54|715065: IP = x.x.x.138, IKE MM Responder FSM error history (struct &0x398d228) <state>, <event>: MM_DONE, EV_ERROR-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent-->MM_SND_MSG2, EV_SND_MSG-->MM_SND_MSG2, EV_START_TMR-->MM_SND_MSG2, EV_RESEND_MSG-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent
7|Jun 23 2010 10:30:52|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:46|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:44|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
6|Jun 23 2010 10:30:44|713905: IP = x.x.x.138, P1 Retransmit msg dispatched to MM FSM
5|Jun 23 2010 10:30:44|713201: IP = x.x.x.138, Duplicate Phase 1 packet detected. Retransmitting last packet.
6|Jun 23 2010 10:30:43|713219: IP = x.x.x.138, Queuing KEY-ACQUIRE messages to be processed when P1 SA is complete.
7|Jun 23 2010 10:30:42|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:38|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:34|713236: IP = x.x.x.138, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:34|715046: IP = x.x.x.138, constructing Fragmentation VID + extended capabilities payload
7|Jun 23 2010 10:30:34|715046: IP = x.x.x.138, constructing ISAKMP SA payload
7|Jun 23 2010 10:30:34|715028: IP = x.x.x.138, IKE SA Proposal # 1, Transform # 0 acceptable Matches global IKE entry # 5
7|Jun 23 2010 10:30:34|715047: IP = x.x.x.138, processing IKE SA payload
7|Jun 23 2010 10:30:34|715049: IP = x.x.x.138, Received DPD VID
7|Jun 23 2010 10:30:34|715047: IP = x.x.x.138, processing VID payload
7|Jun 23 2010 10:30:34|713906: IP = x.x.x.138, Oakley proposal is acceptable
7|Jun 23 2010 10:30:34|715047: IP = x.x.x.138, processing SA payload
7|Jun 23 2010 10:30:34|713236: IP = x.x.x.138, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 104
7|Jun 23 2010 10:30:30|713236: IP = x.x.x.138, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:22|713236: IP = x.x.x.138, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 108
7|Jun 23 2010 10:30:22|715046: IP = x.x.x.138, constructing Fragmentation VID + extended capabilities payload
7|Jun 23 2010 10:30:22|715046: IP = x.x.x.138, constructing ISAKMP SA payload
7|Jun 23 2010 10:30:22|715028: IP = x.x.x.138, IKE SA Proposal # 1, Transform # 0 acceptable Matches global IKE entry # 5
7|Jun 23 2010 10:30:22|715047: IP = x.x.x.138, processing IKE SA payload
7|Jun 23 2010 10:30:22|715049: IP = x.x.x.138, Received DPD VID
7|Jun 23 2010 10:30:22|715047: IP = x.x.x.138, processing VID payload
7|Jun 23 2010 10:30:22|713906: IP = x.x.x.138, Oakley proposal is acceptable
7|Jun 23 2010 10:30:22|715047: IP = x.x.x.138, processing SA payload
7|Jun 23 2010 10:30:22|713236: IP = x.x.x.138, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 104
4|Jun 23 2010 10:30:18|713903: IP = x.x.x.138, Error: Unable to remove PeerTblEntry
3|Jun 23 2010 10:30:18|713902: IP = x.x.x.138, Removing peer from peer table failed, no match!
7|Jun 23 2010 10:30:18|713906: IP = x.x.x.138, sending delete/delete with reason message

"

appreciated.

Alex

Marcin Latosiewicz · ‎06-25-2010

Alex,

I would seriously consider removing the overlapp.

-----------

crypto map outside_map 20 match address outside_cryptomap_20
crypto map outside_map 20 set peer xx.xx.198.138
crypto map outside_map 20 set transform-set ESP-3DES-SHA
crypto map outside_map 20 set security-association lifetime seconds 86400
crypto map outside_map 60 match address outside_cryptomap_60_1
crypto map outside_map 60 set peer xx.xx.198.138
crypto map outside_map 60 set transform-set ESP-DES-SHA
crypto map outside_map 80 match address outside_cryptomap_80
crypto map outside_map 80 set peer xx.xx.198.138
crypto map outside_map 80 set transform-set ESP-3DES-SHA

------------

You con't need all of them! All the entries look the same!.

I would DEFINETLY add a tunnel-group

----------

tunnel-group xx.xx.198.138 ipsec-l2l

tunnel-group xx.xx.198.138 ipsec

pre-shared-key....

--------

As soon as possible upgrade the ASA to a newer release then 7.0.5. It's ancient and 7.0 is no longer continued.

DTSCanada · ‎06-25-2010

Thanks Marcin,

I will remove those entries and those were added when I started having these

problems and I tried to add a new l2l a tunnel for fixing problem.

I already have a L2L tunnel setup, do I still need add following?

"tunnel-group xx.xx.198.138 ipsec-l2l

tunnel-group xx.xx.198.138 ipsec

pre-shared-key....

"

by the way,

I once tried to remote access asa behind the l2l vpn and was trying to setup a RA VPN-Cisco vpn client ( from the same IP) to connect ASA, then the gateway already has a site2site vpn to other end and then another remote client vpn is setting up a nother vpn tunnel from the same IP, will this be the cause for the drop. FYI, 2 VPN can connect at the same time when ok and when VPN dropped, they both dropped.

any thought on this?

Best Regards,

Alex

Marcin Latosiewicz · ‎06-25-2010

Alex,

Yes please setup a separate proper tunnel-group not to let the tunnel land on default one like it does now.

Can you plase re-phrase this?

"I once tried to remote access asa behind the l2l vpn and was trying to setup a RA VPN-Cisco vpn client ( from the same IP) to connect ASA, then the gateway already has a site2site vpn to other end and then another remote client vpn is setting up a nother vpn tunnel from the same IP, will this be the cause for the drop. FYI, 2 VPN can connect at the same time when ok and when VPN dropped, they both dropped."

I'm getting a headache reading

Marcin

DTSCanada · ‎06-25-2010

Sorry for the confusion.

I mean: I setup a site2site between 2 gateways and behind the gateway, I

also installed cisco vpn client and created a remote ipsec vpn tunnel to

other gateway, which is the same ip address to the other end(using nat). 2

vpn work together and down together.

will this give me the problem?

hope this will help.

Best Regards,

Alex

Marcin Latosiewicz · ‎06-26-2010

Alex,

Gotcha now.

Well from debugs it looks like it's nothing related to ASA itself. Some IKE messages are never received by ASA, be it rate limiting or some dynamic filtering.

During the problem what is a result of extended ping from one public IP to the other (say 10000 echo requests with timeout of 1)? (Provided that it works OK during normal operation)

There's still quite a few questions mark here, but unless we can explain where and why the packets are being dropped there's not much we can do.

ASA and PIX both have capture machenism build in.

You can capture for IKE (udp/500) packets during the issue to see if ASA is truly sending the IKE packets and not receiving anything back. But this is just to confirm our observation.

Marcin

agrigoldindia9999 · ‎10-07-2011

Hi All,

Any update on this issue ?

because iam also faceing same issue from last few days......

iam also haveing the same setup and devicess useing ASA5520, RV082 for site to site.

on every day peak hours my remote vpn users are faceing 412 remote gateway not respoinding .

Regards,

Prasad