ASA5545-X dropping packets on inside and outside interfaces
I worked remotely with one of our clients to get a new ASA5545-X installed for them yesterday because they were having internet connectivity issues and their old 5520 was getting pounded. However, this didn't fix their internet connectivity issues and we are fairly certain it's not this new ASA. The kicker with this issue is, is that it happens when they approach 12,000 connections through the ASA. This was one reason we thought it was the old ASA as the 5520 gets crushed with that many connections. They have an odd setup for how their data gets to the internet. Here is how the traffic flows:
We took the iPrism out of the mix and still had issues. The client is going to call Comcast and FatPipe as well. If I look at the inside and outside interfaces, both are constantly showing dropped packets:
Interface GigabitEthernet0/0 "OUTSIDE", is up, line protocol is up Hardware is i82574L rev00, BW 1000 Mbps, DLY 10 usec Auto-Duplex(Full-duplex), Auto-Speed(1000 Mbps) Input flow control is unsupported, output flow control is off MAC address f44e.059e.f430, MTU 1500 IP address 173.15.X.X, subnet mask 255.255.255.240 78542888 packets input, 77587589850 bytes, 0 no buffer Received 33737 broadcasts, 0 runts, 0 giants 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 pause input, 0 resume input 0 L2 decode drops 61894274 packets output, 26309754996 bytes, 0 underruns 0 pause output, 0 resume output 0 output errors, 0 collisions, 1 interface resets 0 late collisions, 0 deferred 4 input reset drops, 0 output reset drops input queue (blocks free curr/low): hardware (465/364) output queue (blocks free curr/low): hardware (446/120) Traffic Statistics for "OUTSIDE": 78542884 packets input, 76113344584 bytes 61894274 packets output, 25031942679 bytes 775210 packets dropped 1 minute input rate 3774 pkts/sec, 4081168 bytes/sec 1 minute output rate 2473 pkts/sec, 445102 bytes/sec 1 minute drop rate, 84 pkts/sec 5 minute input rate 3801 pkts/sec, 4044740 bytes/sec 5 minute output rate 2728 pkts/sec, 520275 bytes/sec 5 minute drop rate, 46 pkts/sec
Interface GigabitEthernet0/1 "INSIDE", is up, line protocol is up Hardware is i82574L rev00, BW 1000 Mbps, DLY 10 usec Auto-Duplex(Full-duplex), Auto-Speed(1000 Mbps) Input flow control is unsupported, output flow control is off MAC address f44e.059e.f42c, MTU 1500 IP address 172.16.X.X, subnet mask 255.255.255.0 58769406 packets input, 23949262559 bytes, 0 no buffer Received 958 broadcasts, 0 runts, 0 giants 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0 pause input, 0 resume input 0 L2 decode drops 81620575 packets output, 79553299593 bytes, 0 underruns 0 pause output, 0 resume output 0 output errors, 0 collisions, 6 interface resets 0 late collisions, 0 deferred 277 input reset drops, 375 output reset drops input queue (blocks free curr/low): hardware (509/370) output queue (blocks free curr/low): hardware (505/307) Traffic Statistics for "INSIDE": 58768288 packets input, 22710337662 bytes 81621115 packets output, 78021353831 bytes 292132 packets dropped 1 minute input rate 2381 pkts/sec, 440472 bytes/sec 1 minute output rate 3737 pkts/sec, 3985284 bytes/sec 1 minute drop rate, 25 pkts/sec 5 minute input rate 2377 pkts/sec, 395251 bytes/sec 5 minute output rate 3671 pkts/sec, 4001053 bytes/sec 5 minute drop rate, 14 pkts/sec
If I do a 'show asp drop,' this is what I see:
Frame drop: NAT-T keepalive message (natt-keepalive) 212 IPSEC tunnel is down (ipsec-tun-down) 20 Flow is denied by configured rule (acl-drop) 421814 First TCP packet not SYN (tcp-not-syn) 113789 Bad TCP checksum (bad-tcp-cksum) 1 Bad TCP flags (bad-tcp-flags) 36 TCP data send after FIN (tcp-data-past-fin) 1 TCP failed 3 way handshake (tcp-3whs-failed) 10823 TCP RST/FIN out of order (tcp-rstfin-ooo) 29204 TCP SEQ in SYN/SYNACK invalid (tcp-seq-syn-diff) 83 TCP SYNACK on established conn (tcp-synack-ooo) 31 TCP packet SEQ past window (tcp-seq-past-win) 6247 TCP invalid ACK (tcp-invalid-ack) 8708 TCP Out-of-Order packet buffer full (tcp-buffer-full) 71898 TCP Out-of-Order packet buffer timeout (tcp-buffer-timeout) 9022 TCP RST/SYN in window (tcp-rst-syn-in-win) 80 TCP dup of packet in Out-of-Order queue (tcp-dup-in-queue) 12998 TCP packet failed PAWS test (tcp-paws-fail) 822 Slowpath security checks failed (sp-security-failed) 147 Expired flow (flow-expired) 1194 DNS Inspect id not matched (inspect-dns-id-not-matched) 1 Dropped pending packets in a closed socket (np-socket-closed) 39
Last clearing: 09:33:43 EDT Oct 23 2014 by admin
Flow drop: Inspection failure (inspect-fail) 562
Last clearing: 09:33:43 EDT Oct 23 2014 by admin
I am doing some other looking into what the issue might be, but I figured I would put this out here to have the community look at to see if others have had this issue. We would like them to get rid of the FatPipe and iPrism, but due to other circumstances, they cannot at the moment.
Thanks for the reply. I opened a TAC case as well and they said the same thing about the tcp-not-syn drops. This is puzzling to me since they only have one internet connection and all the routing is static. The only thing I could figure is maybe it's something with the fatpipe device since it is routing traffic between the ASA and Comcast cable modem. The client is going to call fatpipe today and I am waiting to see when the issue happens again to gather information for the TAC engineer.
DocumentationCode download linksGoalRequirementLimitationsSupported ISR
and UCS-E ModelSupported ISRG2 and UCS-E Blades:Supported ISR4K and
UCS-E Blades:Step by Step ConfigurationConfigure one of the connectivity
options to access the Cisco IMC from the n...
Firepower Threat Defense (NGFWv) on UCS E-series - Transparent Mode in
HA DocumentationCode download linksGoalRequirementLimitationsSupported
ISR and UCS-E ModelSupported ISRG2 and UCS-E Blades:Supported ISR4K and
UCS-E Blades:Step by Step ConfigurationCo...
Question I am currently unable to specify "crypto keyring" command when
configuring VPN connection on my cisco 2901 router. The following
licenses have been activated on my router :