Re: Cisco ASA AnyConnect DDoS Protection

Bernd Nies · ‎03-26-2024

Hi,

Our Cisco ASA running on Firepower 2120 hardware are being targeted by botnet DDoS doing random unauthorized login attempts originating from TOR network exit points or infected webservers and devices all over the world. The syslog is flooded with:

Mar 26 11:20:47 fw1-web2 : %ASA-6-113010: AAA challenge received for user postmaster from server 10.0.x.x.
Mar 26 11:20:48 fw1-web2 : %ASA-6-113005: AAA user authentication Rejected : reason = Memory error : server = 10.0.x.x : user = postmaster : user IP = a.b.c.d

This causes also good logins from real users to fail. Cisco TAC recommends this document:

https://sec.cloudapps.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-asaftd-ravpn-auth-8LyfCkeC

What I've done so far:

Updated ASA on Firepower 2120 to latest release ASA 9.18(3)56 and ASA 5516-X to 9.16(4)55.
Disabled the default group-policies and tunnel-groups as recommended in above document.
Applied access-list on the control-plane to deny access from TOR network and list of malicious IP ranges
Wrote a Python script to update the object-groups on all ASA firewalls.

object-group network TOR_Network_Exitpoints
 description Last updated: 2024-03-18 06:06:02.848267
 network-object host 2.57.122.58
 network-object host 2.57.122.215
 network-object host 2.57.122.246
 network-object host 2.58.56.43
 network-object host 2.58.56.90
····················8<····················

object-group network VPN_Blacklist
 description Updated from /usr/local/etc/asa_vpn_blacklist.txt on 2024-03-23 11:37:46.535348
 network-object 91.92.251.0 255.255.255.0
 network-object 94.232.47.0 255.255.255.0
 network-object 91.108.241.0 255.255.255.0
 network-object 91.92.240.0 255.255.240.0
 network-object 62.122.184.0 255.255.255.0
····················8<····················

access-list control-plane_access_in extended deny ip object-group TOR_Network_Exitpoints any
access-list control-plane_access_in extended deny ip object-group VPN_Blacklist any

access-group control-plane_access_in in interface outside control-plane

https://www.cisco.com/c/en/us/support/docs/security/adaptive-security-appliance-asa-software/217679-asa-access-control-list-configuration-ex.html#toc-hId-176617799

However, this works only until there are attacks coming from new source IP addresses. While auth flood attack is going on, good users get a "Login failed" on AnyConnect client and can't connect to VPN.

Are there other ideas and tricks how to block this kind of DDoS ... except replacing all ASA with FTD?

Regards,

Bernd

tvotna · ‎03-26-2024

Replacing ASA with FTD won't help as the two products are almost identical when it comes to VPN.

Try "authentication certificate" in default tunnel groups:
https://community.cisco.com/t5/vpn/how-to-disable-defaultwebvpngroup-in-asa-5512-x/m-p/5010808/highlight/true#M293407

This may not stop attacks, if attackers know tunnel-group names on your firewall and those tunnel-groups are configured with "authentication aaa". In this case you may need to either implement cert+aaa authentication everywhere or filter out malicious connection attempts on a perimeter device, if it is capable of TLS/JA3 filtering. Follow links in this post:
https://community.cisco.com/t5/vpn/fmc-ftd-webvpn-password-spray/td-p/5050206

Finally I'd contact psirt@cisco.com because of two reasons: a) their recommendations in the mentioned advisory do not work (and don't even mention "authentication certificate") b) the below message is a clear indication of a software failure due to a DoS attack, despite correct and recommended firewall configuration (notice "memory error"):

%ASA-6-113005: AAA user authentication Rejected : reason = Memory error : server = 10.0.x.x : user = postmaster : user IP = a.b.c.d

Finally, TAC must provide explanations what causes this "memory error".

Bernd Nies · ‎03-26-2024

Can't use certificate based authentication for AnyConnect in our case as we have to use multi-factor authentication via RADIUS and an app on the smartphone and we have also have to support AnyConnect on non-managed private devices and from supporters with managed notebooks by their support company.

tvotna · ‎03-26-2024

You may not need to use cert auth.

I guess 99% of attacks are just connection attempts to FQDN. In this case connection lands to DefaultWEBVPNGroup, unless "group-url https://vpn.company.com" is explicitly configured in some other tunnel-group. Then attacker either tries to authenticate to the tunnel group he connected to, or spoofs tunnel-group name to either DefaultWEBVPNGroup or DefaultL2LGroup or DefaultADMINGroup. In the case of the last two the connection attempt should fail, if the firewall runs newer code with the vulnerability fixed.

This basically means that configuring "authentication certificate" in just the DefaultWEBVPNGroup (or in the group with the "group-url https://vpn.company.com") should block 99% of password-guessing attacks. Legit users should connect to another tunnel-group with "group-url https://vpn.company.com/somethingelse". Mapping by group-url is of course preferred over group-alias, which is obviously insecure.

Bernd Nies · ‎03-26-2024

So you mean something like this?

aaa-server AAA_Sinkhole protocol ldap



group-policy DfltGrpPolicy attributes
 vpn-simultaneous-logins 0

group-policy company_policy1 internal
group-policy company_policy1 attributes
 vpn-simultaneous-logins 5

group-policy company_policy2 internal
group-policy company_policy2 attributes
 vpn-simultaneous-logins 5



tunnel-group DefaultRAGroup general-attributes
 authentication-server-group AAA_Sinkhole
tunnel-group DefaultRAGroup webvpn-attributes
 authentication certificate
 
tunnel-group DefaultWEBVPNGroup general-attributes
 authentication-server-group AAA_Sinkhole
tunnel-group DefaultWEBVPNGroup webvpn-attributes
 authentication certificate

tunnel-group company_group1 type remote-access
tunnel-group company_group1 general-attributes
 authentication-server-group AAA_Group1
 authorization-server-group AAA_Group1
 accounting-server-group AAA_Group1
 default-group-policy company_policy1
 authorization-required
tunnel-group company_group1 webvpn-attributes
 radius-reject-message
 group-alias "Company Group1" enable
 group-url https://vpn.company.com/group1 enable

tunnel-group company_group2 type remote-access
tunnel-group company_group2 general-attributes
 authentication-server-group AAA_Group2
 authorization-server-group AAA_Group2
 accounting-server-group AAA_Group2
 default-group-policy company_policy2
 authorization-required
tunnel-group company_group2 webvpn-attributes
 radius-reject-message
 group-alias "Company Group2" enable
 group-url https://vpn.company.com/group2 enable

Bernd Nies · ‎03-26-2024

Seems setting authentication certificate in the default tunnel-groups did the trick. I could observer some attacking IPs contacting the ASA, but then disconnecting again without AAA flood. Thank you!

Edit: oh, was too early happy. Still getting the AAA Memory errors.

Bernd Nies · ‎03-27-2024

I tried both with only AAA sinkhole and with only certificate and with both.

aaa-server AAA_Sinkhole protocol ldap  

tunnel-group DefaultRAGroup general-attributes
 authentication-server-group AAA_Sinkhole
tunnel-group DefaultRAGroup webvpn-attributes
 authentication certificate
 
tunnel-group DefaultWEBVPNGroup general-attributes
 authentication-server-group AAA_Sinkhole
tunnel-group DefaultWEBVPNGroup webvpn-attributes
 authentication certificate

So far this kind of AnyConnect VPN attack does not care. Occasionally there are floods with "AAA user authentication Rejected : reason = Memory error" and causes login failure of good users until I blacklist the source IP on the control-plane.

I don't know FTD yet, but I would expect that one could subscribe to a database of malware IP addresses and add that one create a rule in the access-list for the control-plane.

Rob Ingram · ‎03-27-2024

@Bernd Nies wrote:

I don't know FTD yet, but I would expect that one could subscribe to a database of malware IP addresses and add that one create a rule in the access-list for the control-plane.

Unfortunately not yet, the control plane functionality on FTD as of 7.4 is the same as ASA. I believe this will be improved in upcoming FTD versions.

giovanni.augusto · ‎03-27-2024

To add to this:

At the moment, AFAIK, the only way to achieve some kind of filtering of traffic directed to the firewall internet zone (and terminated on it, like RA VPN) is well...to add another firewall in front of it!

FTD has indeed a security intelligence database for malicious IPs and also a geolocation feature for IPs and both are updated quite regularly but it works only at the data plane level (so only if traffic is transiting two zones, which would be the case if you just stick a firewall in front of your RA firewall).

That is why, in the OP's case I recommended a cloud edge solution to stick to the on-premises firewall

tvotna · ‎03-27-2024

Well, if you have "tunnel-group-list enable" under "webvpn" and "group-alias" in tunnel-groups, you're exposing all of your tunnel-groups by their alias names to the outside world. It's always recommended to use AnyConnect profiles and binding by "group-url" instead of binding by "group-alias". User experience won't change much. Users will see connection URLs from the profiles, like "https://vpn.company.com/group1" instead of "group-alias" drop down list.

Unfortunately, it may not be easy to understand which tunnel-group is a target and what exactly happens. There is a "debug aggregate-auth xml 255", but it generates lots of messages and should be used only when the number of connection requests is low.

giovanni.augusto · ‎03-26-2024

I think you would need DDoS protection service such as what cloudflare offers:

https://www.cloudflare.com/network-services/products/magic-transit/

There are others but just to give you an idea

Bernd Nies · ‎04-14-2024

The current botnet attack in the wild originates from infected websites running Cpanel and Wordpress. It connects to port 443 and tries to authenticate with the first tunnel-group shown (listed alphabetically) when webvpn tunnel-group listing is enabled. In our case when one can't disable tunnel-group listing for user conveniene, the workaround is creating a Bot Trap that is shown first and send those authentication attempts to a nonexisting AAA server.

tunnel-group DefaultRAGroup general-attributes
 authentication-server-group AAA_Sinkhole

tunnel-group DefaultWEBVPNGroup general-attributes
 authentication-server-group AAA_Sinkhole
tunnel-group DefaultWEBVPNGroup webvpn-attributes
 group-alias "0. Bot Trap - Ignore" enable

tunnel-group group1 type remote-access
tunnel-group group1 general-attributes
 [...]
tunnel-group group1 webvpn-attributes
 radius-reject-message
 group-alias "1. Group One" enable
 group-url https://vpn.example.com/group1 enable

tunnel-group group2 type remote-access
tunnel-group group2 general-attributes
 [...]
tunnel-group group2 webvpn-attributes
 radius-reject-message
 group-alias "2. Group Two" enable
 group-url https://vpn.example.com/group2 enable

tunnel-group group3 type remote-access
tunnel-group group3 general-attributes
 [...]
tunnel-group group3 webvpn-attributes
 radius-reject-message
 group-alias "3. Group Three" enable
 group-url https://vpn.example.com/group3 enable

aaa-server AAA_Sinkhole protocol ldap

In environments where one has certificate deployment to managed user devices, one could switch to certificate based authentication and predeployed AnyConnect profiles. But when one has to allow AnyConnect access for private and subcontractor devices, this is a no go.

Bernd Nies · ‎04-16-2024

There are already some enhancement requests for DoS protection:

Rob Ingram · ‎04-17-2024

@Bernd Nies cisco released a guide last week which might be of some help:-

Implement Hardening Measures for Secure Client AnyConnect VPN

Bernd Nies · ‎04-17-2024

I tried it once with disabling tunnel-group listing and the AnyConnect profile and listing all group-urls in <ServerList><HostEntry> sections. With multiple VPN peers (one per location) and multiple tunnel-groups (one per authentication method) this renders a long list in the AnyConnect URL dropdown but it shows only the first 4 ones.