I have a problem with a customer trying to reach a server pbo-prd01 in our network over a VPN from ip address 'cust-prd01'where the packets seem to be dropped by 'Flow closed by inspection'
What does this mean and how can i fix it?
The IPS sensor in the firewall is switched off.
cust-prd01 18580 pbo-prd01 1860 Teardown TCP connection 17266 for External:cust-prd01/18580 to Inside:pbo-prd01/1860 duration 0:00:08 bytes 11007 Flow closed by inspection
cust-prd01 18580 pbo-prd01 1861 Teardown TCP connection 17268 for External:cust-prd01/18580 to Inside:pbo-prd01/1861 duration 0:00:07 bytes 8847 Flow closed by inspection
cust-prd01 18580 pbo-prd01 1862 Teardown TCP connection 17270 for External:cust-prd01/18580 to Inside:pbo-prd01/1862 duration 0:00:06 bytes 8393 Flow closed by inspection
Are you inspecting the http traffic? Can you post the output of "show run
policy-map" and "show service-policy" commands?
I dont believe http traffic is being inspected...
cust-vpn-fw01# show run policy-map
policy-map type inspect dns preset_dns_map
message-length maximum 512
inspect icmp error
flow-export event-type all destination 172.29.0.158
cust-vpn-fw01# show service-policy
Inspect: icmp error, packet 0, drop 0, reset-drop 0
Is it affecting your traffic? If it is not affecting the traffic, you might
be hitting a bug where the ASA is generating the log message incorrectly
even when the flow is terminated normally. The bug ID is CSCtg17779.
Yes, traffic is being affected. I see the XML request leave but the return packet is dropped by this issue.
pbo-prd03 2632 x.x.x.x 18580 Built outbound TCP connection 24127 for External:x.x.x.x/18580 (x.x.x.x/18580) to Inside:pbo-prd03/2632 (pbo-prd03-nat/2632)
x.x.x.x 18580 pbo-prd03 2632 Teardown TCP connection 24127 for External:x.x.x.x/18580 to Inside:pbo-prd03/2632 duration 0:00:04 bytes 9943 Flow closed by inspection
Can you sniff the traffic on your inside to see if the server is closing the
connection? If the server closes the connection via FIN or RST, the firewall
will also close the connection but due to the bug I have mentioned, it will
wrongly generate the "flow closed by inspection" message. As mentioned in
the bug, the flow will be terminated for other reasons (normal firewall
operations) but the error message will be different. The sniffer capture
will be useful in determining the root cause of the flow termination.
I've run a wireshark trace on the firewall and the connection is closes down ok
internal server --> external server FIN,ACK.
external server --> internal server ACK
external server --> internal server FIN, ACK
internal server --> external server ACK.
I'm not sure it's that bug causing problems.
So, both devices exchange FIN/ACK indicating that they are done with data
transfer. This indicates to the firewall that the connection is no longer
needed and it can tear-down the connection.
I have seen those kind of errors when the vpn that the traffic is going over drops, even for a second. When the tunnel goes down, the conns going over it are torn down and cite that reason. You could try adding the following two command to prevent the connection from being removed instantly: Sysopt connection preserve-vpn-flows Sysopt connection reclassify-vpn The first one keep the connection if the tunnel drops and the second one re encrypts should the tunnels come back up. - Magnus
Hmm I believe I have seen this issue before.
I think what is really happening is that the connection is timing out and so the inspection is closing due to timeout.
When I saw it, it was due to a policy-nat statement conflicting with the nat statement required, but the fact that your wireshark shows traffic hitting the server would seem to disprove this theory, perhaps something to check though.
All these seem to be very short lived (6,7 and 8 seconds) connections.
Issue this command and see what inspections are being hit and see if you can deny this flow from being inspected.
sh service-policy flow tcp ho cust-prd01 ho pbo-prd01 eq 1862
Hi I've tried this command but it just comes up blank, no stats.
Any other ideas? I cant see how a firewall configured with no inspection drops packets? Doesn't seem logical, but I def see packets leaving the external interface but getting dropped on the way back. Might have to open a TAC