URL server request dropped in ASA

[hxmengmetro]

Hi All,

"Request dropped" in url-server statistics. What does that mean?

As  I understand, it means the url lookup request sent from ASA to websense  got dropped since no response or time out from websense server? Is this  correct? But how about the other counter "Server Timeouts"? Is there  any relation between these two counters? It seems "server timeout"  doesn't have to cause the dropped request from what I'm seeing.

Thanks a lot.

Lou

[Maykol Rojas]

Hello Hailu,

I hope you are doing great,  Sorry I did not reply on the other post, Basically the request drop means that the firewall dropped the request from one of the host because he did not hear back from the websense......maybe he was overwhealmed at the time he got the request and was not able to respond in a timely manner.

If you have configure url-block block that will tell the firewall to create a queue and if he doesnt get a response back from the websense by the time the queue is full, you will be able to see a hit incrementing on the request dropped.

As a troubleshooting step you can try removing the url-block block command from the configuration (if present) to see if that solves the issue.

Cheers

Mike

[hxmengmetro]

We setup 128 block. The statitics is really ugly:

URL Pending Packet Buffer Stats with max block  128
-----------------------------------------------------
Cumulative number of packets held:              253169
Maximum number of packets held (per URL):       8
Current number of packets held (global):                3
Packets dropped due to
       exceeding url-block buffer limit:        160676
       HTTP server retransmission:              69022
Number of packets released back to client:      227613

It didn't have the block configuration before. I added the block configuration since we were getting the request drop issue. After adding this, the issue still doesn't go away. The websense tech advise our websense box is not 4 core. only 2 core. The design docs shows 4 core is recommended. I saw the websense cpu usage is below 60 in normal operation except when we update the websense database. Here are my other configuration in the box:

url-server (inside) vendor websense host websense_ip timeout 30 protocol TCP version 4 connections 5

url-block url-mempool 1500
url-block url-size 4
url-block block 128

some filter configurations...

As I understand without block configuration, the requst is more possible to be dropped, right?

Thanks.

Lou

[Maykol Rojas]

Hello Lou,

Not in your particular case no, the problem that you are having is related to timeout with the server, and also the problem with the request dropped, if you put a block over there, you are forcing the ASA to wait until he hears back from the websense (since it is TCP based), knowing that you are also having timeout problems, this will make the problem worst.

As a troubleshooting step, is it possible for you to change the communication to UDP instead of TCP?

Let me know.

Mike

[hxmengmetro]

Thanks Mike. I can remove the block configuration now. For UDP configuration, do I need remove the

url-block url-mempool 1500
url-block url-size 4

Since this is for TCP only, the UDP configuration will not utilize the above settings, right? I can do this UDP change after work when everybody is off work. We had issue two weeks ago. The websense server is totally showing "down" status in ASA. At that time, we changed to UDP and the connection got restored. Then we changed it back to TCP since UDP can't support some feature like long URL. It may cause other issues. Is there any other potential risk to use UDP?

Thanks a lot!

Lou

[hxmengmetro]

Hi Mike,

I removed block configuration. I'm capturing the packets right now. So far no request drop happened. Lots of time out and retries. I found when the time out happened, the capture shows FIN ACK packet which terminates the current TCP connection. Since we set up 30 seconds timeout time, I went back 30 seconds in capture, but there is nothing abnormal at that time. I hope i can catch that some time later.

Lou

[Maykol Rojas]

Hello Lou,

Lets keep monitoring this behavior and check what happens.

Mike.

[hxmengmetro]

Mike,

I kept monitoring the system closely and found ASA popped up the log "URL server is not responding" and at the same time, the requests dropped increased. The URL server status in ASA is "DOWN". It was actually bouncing at that time. I'm thinking the reason for the dropped request is "URL server is not responding". The reason causing this is unknow at this time. The capture only shows FIN ACK packet at that time.

Lou

[apothula]

Hi Lou,

To check for the responsiveness of the URL filtering server, you can run a ping to the server from the ASA and see the response statistics and response times.

Alternately, you can apply captures between the ASA and the URL filtering server and check for activity there as well.

Please find  below, the guide to setting up captures on the Cisco ASA,

http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a00807c35e7.shtml

Cheers,

Nash.

[hxmengmetro]

Thanks Nash. I did get some capture for the traffic between ASA and websense but I didn't catch anything wrong. Since the capture buffer size is really small. I used circular buffer to capture. I need always watch the request drop count and then stop the capture if the number goes up.

Lou

[Maykol Rojas]

Hello Lou,

Where you able to check Url-block configuration?

Let me know.

Mike