We’re seeing a situation right now that seems to happen right before our firewall reboots itself. This is an ASA5540 running 8.2(1)11 code in an active/passive failover setup.
1. The firewalls CPU is around 50%, and is normally around 4%
2. The IPS modules CPU is stuck around 100% and inspection load stays around 70-90%, both are usually much lower
3. The firewall dashboard in ASDM shows a VPN connection as the source of most of the current traffic
4. When I do a “sh connection address” (and the address of the connection from the firewall dashboard), it shows it as currently being connected
5. If I try to clear the connection using the "clear connection" command, it does not clear
6. If I try to clear it using the "clear local-host" or "clear xlate" commands, it still will not clear the connection
7. If I manually failover the firewall, I can then clear the connection, manually fail it back over, and everything returns to normal
8. If I do nothing, the primary firewall will eventually reboot itself, failing over to the secondary unit and clearing the connection
Has anyone ever seen anything like this before? We're stumped and Cisco TAC hasn't been able to figure it out yet.
That connection that you can't kill, is it a legal/valid connection?
I mean it comes from the outside with a valid public IP to a valid destination port on the inside? Or what sort of connection is it?
The reason I ask this, is because could be part of an attack.
Try capturing the output of the ''show connection IP'' for that specific connection and post it.
The connection is a valid VPN client connection. The weird thing is when I look at the current VPN connections, that connection shows as not being connected. But if I use the "sh connection" command, it does show as being connected.
You see that connection when doing all the following commands?
For that IP?
Besides being a VPN client connection, what kind of traffic is it? TCP/UDP port numbers?
Well, that seems normal since traffic through the VPN tunnel will not create an XLATE unless using NAT....
Besides that, is there a change or something that you've done to increase the processing on the ASA that much?
This behavior happens with any VPN client connecting from any location or just this particular one?
We haven't made any changes on the firewall for the past 2 weeks.
We have many users connecting through the VPN and this seems to happen at least once a week. It's not any particular user or day or time. It's like the connection get's stuck in the firewall and we can't get rid of it unless we failover to the secondary firewall and then clear the connection. When we fail back over to the primary, the CPU is back down to 4-5% and everything is happy until it happens the next time.
I believe that's the way to go....
Since you have a failover pair, you can do a zero downtime upgrade for both units...