TCP out-of-order at IPS

[Anuar Shahrin]

Dear All,

We have a setup the IPS 4510 working inline mode with strict inspection turn on. we have detected some latency issue accessing the internal website. So we did some capture at the IPS interface. We found that there's a lot of out-of-order packet and DUP ACK detected by IPS which causing the normalizer engine buffer full and could not handle anymore request. As a work around we put the IPS in asymmetric mode where it turn off the IPS normalizer engine. 

I need some opinion on possibilities why the Out of order and DUP ACK happen. 

We are seeing quite a lot of Out-of-order, DUP ACK and TCP zero window in TCP stream that we captured. 

The topology is quite straight forward:

Internet ----WAN ROUTER ----- IPS4510 ----- ASA ----- Web server

There's no redundancy or load balance for the ASA or WANROUTER. 

Im hoping for some opinion and idea on how to tackle this issue.

Thank you very much

[mhnedirli]

Hello, 

For which traffic do you have this problem, did you try any other tcp session for same ip and port. Could you share ASA logging for this traffic. generally firewalls block tcp traffic when it gets out of order packets you can listen traffic with wireshark on server site and trace tcp traffic. 

[Anuar Shahrin]

Hi,

Thanks for the reply.

Unfortunately Im unable to access the ASA right now. 

We did the capture at the IPS where we are seeing the out-of-order packet. 

You may refer the capture. Unfortunately i could not show the IP. The IP is a public IP which we did the nat at ASA from our internal network. 

Basically at the idea of the IP is as below:

2.2.2.2 ---- ASA NAT ------ F5 BIG-IP :192.168.100.100---- Web Accelerator : 192.168.100.20( it has 4 itnerface connected to the server with the same range of IP and Gateway to ASA FW.

[mhnedirli]

Hi, 

Could you check F5 session table, in the middle a device blocking your SYN+ACK packet. 

[Anuar Shahrin]

Hi, 

the F5 is deployed in one-arm configuration. 

We also suspecting the same. 

Is there any problem if we have a 100Mpbs connection between ASA and IPS while others are in 1000Mbps?

[mhnedirli]

Hi,

Yes, check the MTU size of devices, maybe IPS site try to send big packet because of MTU size, other device in the network will try to fragment this packet, maybe ASA etc. blocks the fragmented packets. Try to change MSS size  for the TCP traffic. 

[Anuar Shahrin]

Hi

bumping out an old thread since the issue still on going.

I already discussed with TAC regarding the issue and 2 option that she gave

+ asymmetric mode (Which we rejected as permanent solution)

+ Event action filter

I'm currently looking at this solution and plan to implement it in the IPS.

I need to consider a few things and also suggestion

+ The signature engine involve is Normalizer engine (specifically sig 1330)

+ is it possible to customize this signature or should we just go for Event action filter?

need opinion and pro and cons of this.

Thanks a bunch