Problems with VPN disconnections after upgrading ASA to address Cisco security advisory
So, as some background, we have been happily running ASA 8.4(3) for some time. But we actually first experienced this problem a while ago when trying to upgrade ASA software version but downgraded back to 8.4(3) again, the problem went away, and it got sidetracked for various reasons (primarily that we intended upgrading the ASA hardware anyway though, typically, that hasn't actually been carried out yet!)
Anyway, following the latest Cisco security advisory:
BUT - the same problems we experienced last time we tried upgrading (on that occasion it was to 8.4.6) were experienced again. Basically, after upgrading, we experience VPN connectivity problems with (at least) one of our sites - the VPN connectivity with them constantly establishes then drops, then re-establishes again. If we return to 8.4(3) everything is fine.
After upgrading, on the Standby ASA, I notice the following messages constantly appearing:
Apr 22 2014 17:27:16: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:16: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:16: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:16: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:17: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:18: %ASA-3-210005: LU allocate connection failed Apr 22 2014 17:27:19: %ASA-3-210005: LU allocate connection failed
When the Standby ASA is made Active (as the "Old Active" is being updated) we then get the following messages for one particular site (we also have several other sites which didn't appear to be affected):
Apr 22 2014 17:27:31: %ASA-3-713061: Group = 22.214.171.124, IP = 126.96.36.199, Rejecting IPSec tunnel: no matching crypto map entry for remote proxy 192.168.1.0/255.255.255.0/0/0 local proxy 188.8.131.52/255.255.255.255/0/0 on interface OUTSIDE Apr 22 2014 17:27:31: %ASA-3-713902: Group = 184.108.40.206, IP = 220.127.116.11, QM FSM error (P2 struct &0x72f2a728, mess id 0xf2924361)! Apr 22 2014 17:27:31: %ASA-3-713902: Group = 18.104.22.168, IP = 22.214.171.124, Removing peer from correlator table failed, no match! Apr 22 2014 17:27:31: %ASA-3-752015: Tunnel Manager has failed to establish an L2L SA. All configured IKE versions failed to establish the tunnel. Map Tag= outside_map. Map Sequence Number = 3002.
Now, this relates to something of a peculiarity we have in our set-up. At the remote site, we split-tunnel internet traffic EXCEPT for traffic to one particular "extranet" site which is routed back through the datacentre ASA and out to the internet through the DC internet connection. (This all works fine on 8.4(3) though)
This makes me think its due to some sort of mismatch between both ends of the connection but, if so, why does it work ok on version 8.4(3)? And the other strange thing is that we have another remote site almost identically configured which doesn't appear to suffer any such problems. The remote site has a Cisco 2911 running 15.1(4)M.
At the time of the issue, the only messages appearing on the Cisco 2911 were:
Apr 22 17:27:31.238: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=126.96.36.199, prot=50, spi=0x95A15F88(2510380936), srcaddr=188.8.131.52, input interface=GigabitEthernet0/0 Apr 22 17:28:31.250: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=184.108.40.206, prot=50, spi=0x4008F420(1074328608), srcaddr=220.127.116.11, input interface=GigabitEthernet0/0 Apr 22 17:32:59.554: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=18.104.22.168, prot=50, spi=0xA198CD52(2711145810), srcaddr=22.214.171.124, input interface=GigabitEthernet0/0
Datacentre - ASA5520 running version 8.4(3), all works fine. If upgraded to e.g. 8.4(6) or 8.4(7.15) then problems occur but only seems to affect one particular site
Remote Site (Office101) - Cisco 2911 running 15.1(4)M
crypto isakmp key **** address 126.96.36.199 ! crypto map cryptomap 10 ipsec-isakmp set peer 188.8.131.52 set transform-set vpn-set match address datacentre ! ip access-list extended datacentre permit ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255 permit ip 192.168.1.0 0.0.0.255 172.16.11.48 0.0.0.15 permit ip 192.168.1.0 0.0.0.255 172.16.10.0 0.0.0.255 permit ip 192.168.1.0 0.0.0.255 172.16.12.0 0.0.0.127 permit ip 192.168.1.0 0.0.0.255 host 184.108.40.206 ! !###NO NAT ACL#### access-list 112 deny ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255 access-list 112 deny ip 192.168.1.0 0.0.0.255 172.16.10.0 0.0.0.255 access-list 112 deny ip 192.168.1.0 0.0.0.255 172.16.11.48 0.0.0.15 access-list 112 deny ip 192.168.1.0 0.0.0.255 172.16.12.0 0.0.0.127 access-list 112 deny ip 192.168.1.0 0.0.0.255 host 220.127.116.11 access-list 112 permit ip 192.168.1.0 0.0.0.255 host [cloud-web-scan-ip] access-list 112 deny ip any any
Has anyone come across similar and/or can suggest what the problem might be? It seems to suggest a misconfiguration but why does it work ok for 8.4(3)? I am concerned now that we cannot address any security vulnerabilities or suchlike because upgrading the ASA version seems to introduce this issue. Also, as I mentioned, we intend upgrading the ASA hardware (to ASA5545x) and, as such, would likely be running 8.6 software on this appliance but I'm worried that introducing the new hardware might also bring in this problem as a result, so I'm naturally keen to find the cause of the issue!
thanks for the response. The permit ip 192.168.1.0 0.0.0.255 host 18.104.22.168 line on the 2911 is to direct traffic to the extranet site (22.214.171.124) through the tunnel to the datacentre ASA (rather than split-tunneling through the 2911's internet connection) and through the datacentre internet connection. So the ACL containing that line is for the "interesting" traffic that will be sent across the VPN tunnel to the datacentre ASA. This seems to work fine with the ASA version 8.4(3)
However, no, I don't have the "sysopt connection preserve-vpn-flows" line - that is interesting, may actually be of benefit? But the thing that would still puzzle me would be why everything works fine at 8.4(3) but only when I upgrade do I experience issue?
it turned out the problem was due to a mismatch in the ACLs between the ASA and the remote site 2911.
We had no matching line in the ASA config for traffic between the office and the extranet server. On the office router, we had:
permit ip 192.168.1.0 0.0.0.255 host 126.96.36.199
So, on the ASA, we had to add:
access-list office101 line 8 extended permit ip host 188.8.131.52 192.168.1.0 255.255.255.0
I'm not sure why the behaviour was different between the ASA versions i.e. why we did not experience any issues when we had the misconfig at version 8.4(3) but it caused problems with higher versions. Also, cannot explain why it only really seemed to be one site that was affected (we had the same config mismatch at other sites which were seemingly alright)
However, ultimately, it was the config mismatch that was causing the problem and, after rectifying that, we were able to upgrade and our remote site VPN connectivity remained stable.
Show Name: Thoughts on Security at Cisco Live US 2018 in Orlando
Contributors: Kevin Klous, David White Jr., Aaron Woland, Jeff Fanelli
Posting Date: June 2018
Description: The team goes on-site in the Cisco Live Speaker room in...
RADIUS and Symantec VIP.
I will use screenshots of ASDM, and at the end I will add the required CLI commands. the diagram below show a diagram of the steps the FW goes through when using 2FA authentication:
As you can see in Fig. 1&nbs...