URL predictor demystify.

[shukla1975]

Dear All,

Have been trying to resolve some issues in ACE for last many months and NetPro have been very helpful,have typical issues and need to understand URL based predictor/hashing with transprent caching to troubleshoot.

1. there few sites which never open in total, meaning some ADs or pictures on the web-page don't open, if we bypass the ACE all works fine.

2. we are not able to justify the increasing "connection failure" counter increment even though we have the latest image which has the "connection failure" bug fixed, we have told the client that servers do send RST in place of FIN which ACE will count as a conn failed, still no hard evidence.

3. The only thing we have confirmed is that the caches will be unevenly loaded which is dependent on internet traffic pattern.

anyone can shed light.

Shukla.

[Gilles Dufour]

If there is a problem to parse the url, you will see an error with the following command :

switch/Admin# show np 1 me-stats "-shttp -v" | i arse

Parse result LB msgs sent: 0 0

Parse result Inspect msgs sent: 0 0

Static parse errors: 0 0

Max parselen errors: 0 0

switch/Admin#

This will kill the connection and prevent object to be loaded.

If you have no error and can't find the source of the problem, capture a sniffer trace on client side (or front of ACE) and cache side (or backend of ACE) while having problem to load an object.

Capture show tech before and after the test.

Send us everything.

Gilles.

[shukla1975]

Hi Gilles,

i have extracted the output from 2 diff ACE setup of same customer:

setup 1 ( ACE-RE):

Parse result LB msgs sent: 381150270 4155

Parse result Inspect msgs sent: 0 0 0

Static parse errors: 166477138 53

Max parselen errors: 126139 0

setup 2 (ACE-RF):

Parse result LB msgs sent: 1836093410 2237

Parse result Inspect msgs sent: 0 0

Static parse errors: 89740805 29

Max parselen errors: 118153 1

have attached the whole output, hope i get some insight through this.

pls let me know

Shukla.

[shukla1975]

Hi Gilles/ NetPro

still awaiting any response from anyone.

Shukla.

[Gilles Dufour]

The max parselen error can easily be fixed by increasing the length with a parameter map.

switch/Admin(config)# parameter-map type http ParseLen

switch/Admin(config-parammap-http)# set header-maxparse-length ?

<1-65535> Enter max-parse length for header

Set the length to 8000 bytes and see if that helps.

You then need to apply this parameter-map to the HTTP policy.

The static parse error could be due to illegal characters in the url. We used to be very strict about that and now we allow them since version A2(1.2)

CSCsm89835

Accept not encoded characters after a ? inside url

Your case was escalated to me internally, so I know you run version A2(1.1).

You should upgrade to A2(1.2)

Gilles.

[shukla1975]

Gilles,

good to hear that you know are case, would be fine if i do a PM to you , as we are still fighting for the same case?..

Should i believe that this could be the reason for our ever increasing "connection failure" counters ( The BIG issue) ?? , the URL we have issue is www.typepad.com , this URL would never finish downloading and would get stuck.

typically the ACE would see the URL till the " ? " mark, with the parselen will it go beyond it ?

So if i upgrade my ACE image and set the parse length at 8000 what am i going to achieve.. ( the answer to this statement will be used to justify to my client for the above changes we need to do !! )

awaiting optimistically.

Shukla.

[Gilles Dufour]

Every parselen error or static parse error will result in a connection failure.

You seem to have a lot of them and they keep increasing.

The upgrade should solve the static parse error (I have no sniffer trace with the error, so can't be 100% sure).

The max parselength should allow the ACE to parse the http header that where exceeding our default parse length.

You could test this solution without upgrading and see if that already fixed some of your problems.

Regards,

Gilles.

[shukla1975]

Hi Gilles/ NetPro team,

This is an urgent request:

due to some reason the www.yahoo.com site is getting blocked after my ISP.

mail.yahoo.com opens except www.yahoo.com itself, its a catastrophic impact on a very large geographic region, if we change the path and bypass ACE, the thing works just fine.

I had made a separate server farm for yahoo.com site and have 8 caches with round robin via matching the "match header" for the url.. its like due to heavy traffic or something from the caches the other ISPs are blocking or denying, need to see tomorrow.

any workaround or insight will be very valuable.

awaiting optimistically

Shukla.

[Gilles Dufour]

Can we get a sniffer trace when going directly (capture on the client).

And another trace captured on the ACE inbound and outbound vlan when it fails.

I want to see the difference.

Thanks,

Gilles.

[shukla1975]

thanks Gilles for showing up,

let me see if i can get a trace, as its operations times now, its tough to ask client for a redirection which will fail.

before that, worked with TAC and advised customer for the maxparselen errors and static errors,

for justification, need to have the websites where the web site is larger than 4096 byte and where the ACE will not compile it (parsing) and drop it, same goes for static errors, so that we can show to client that u have problem, till now the errors are showing not any complaint for site, except what i mentioned.

will be in touch with you shortly,

pls be on the netpro.

Shukla.

[shukla1975]

Dear Gurus/Gilles,

Now after the Max parse length we increased AND we also allowed the URL which exceed them, we "still" see the errors in the max parse lenght, another blow !!.

We are going for image upgrade next week early but its tough justify and explain to client.

Now the issue with yahoo.com is debated to be of load than ACE issue, but we have another one (phew ! ) with Microsoft.

now http://www.microsoft.com works just fine with ACE..

But http://microsoft.com just will NOT open.

This second URL has 2 separate IPs ( 207.46.232.182 , 207.46.197.32 ) .. if i browse with the IP x.x.x.182 i CAN browse but x.x.x.32 will never open the page , has no error and the page never loads.

We also took a capture from the ACE too, filtered it and the client IP is:86.97.91.138

just wanna know what happens to this site , issue with user, ACE, web servers, URL redirection .... ??

any inputs from any other customer issues regarding intermittent URL handling, will sincerely appreciate.

Do share your experiences.

Cheers,

Shukla

[Gilles Dufour]

WE see ACE correctly spoofing the request and trying to open a backend connection with the cache.

The cache never responds to the SYN.

This is a cache issue.

Find out why the cache does not accept the connection.

Does it have a bypass list like cisco cache engines ?

Does it have a 2nd interface ?

Gilles.

[shukla1975]

Hi Gilles,

Appreciate your inputs, before i go to the client with the cache complaint im hereby attaching the 2 zip files which Any-Src_2_Dst.zip which means the traffic going to http://microsoft.com with its 2 IPs:

207.46.232.182,207.46.197.32

And second file is : Any-Src_6_Dst.zip for the traffic going to http://www.microsoft.com and its 6 IPs: 207.46.19.254, 207.46.192.254 ,207.46.193.254, 65.55.12.249, 65.55.21.250 , 207.46.19.190.

Source : ANY.

Few concerns:

we made the max-parse length to 9000 from 4096 and "Also" have the " length continue" command which means even if the length is way beyond 9000 bytes, to continue parsing, still we see errors on the NP-1, this places us in very uncertain position, the client says " why do we still have errors" ? , before which we could not correlate these errors to actual sites failing , still we are moving ahead.

I hope these captures would give some light , we have 2 weeks for the ACE issue which we installed 8 months back before its called a wrap up.

The caches are dual NICs but only 1 is used per cache, the capture was taken on the ingress 2 vlans plus 4 cache vlans .

I do not understand the bypass issue , how can we check it with Bluecoat guys ?

Do we have known issues with Bluecoat caches ?

Do we have any issues with normalization being enabled or disabled ?

awaiting optimistically

Shukla.

[shukla1975]

Hi Gilles,

have upgraded the image to A2(1.2) and changed max-parse length to 9000 and also issues the command "length continue", still we see errors in both the cases (static parse and parse-length).. :-(

myself and TAC have found out that cache gives "503 service unavailable error" , but from URL website to cache i get "302 found" , will test this today ,

1.Is cache able to reach Microsoft but giving 503 to client is to be verified ?

Shukla.

"will not go down without a bout."

[shukla1975]

Dear All,

Any updates or i shall close this post.

Shukla