CSS Basic LB Question

jkriss360 · ‎09-11-2009

We have configured our CSS to do basic Roundrobin load balancing between 3 webservers. Occasionally we notice that between 1/3 and 1/2 of the requested page content loads and then the browser gives a page cannoot be loaded response. A refresh then loads teh page correctly and quickly.

I believe I can try to sort out what the CSS is doing by using logging. I have set

logging as follows....

Syslog Hosts:

IP Address: 10.9.x.x

Facility: 1

Logs Sent: 278

Log Level: 7

I am seeing some log entries but no where near what I would expect where each connection (GET request) would generate a log entry.

Can someone help and let me know if I am correct in my assumptions on how logging works or if there is a way to see all activity on teh CSS.

Thanks

jason.espino · ‎09-11-2009

Hello,

Do you wish to log all subsystems on the CSS? Or have you enabled logging at level-7 for a specific subsystem?

As for the balancing issues you are experiencing do you have any type of session persistence enabled on the content rule the 3 webservers are a member of? The time that 1/3, or 1/2 of the connections are unable to load are you able to pinpoint what destination server the connections landed on? During these load issues do any of the services register a state transition?

- Jason

jkriss360 · ‎09-11-2009

Thanks Jason, I managed to find the problem it was actually a problem with one of teh webservers.

Meanwhile, I'd still like to log all subsystems and I cannot get that to work.

Also because I cannit seem to get logging to work, I cannot really tell where the request is being forwarded to.

Any help you can suppy is appreciated.

jason.espino · ‎09-11-2009

Hello Jeff,

Glad to see the server issue has been resolved!

As for the logging on the CSS, do you have the following command setup to allow logging of all subsytems?

logging subsystem all level debug-7

In all honesty you really don't need to log everything within each subsystem. If you wish to see when a service experiences a state transition (Alive, Down, Suspended) the following log command should suffice:

logging subsystem netman level notice-5

When a service changes it's state to either "Down" or "suspended" the CSS will generate a log using the netman subsystem at level 2 (critical). However, if the service passes it's keep-alive check again the CSS will generate a log entry using the same netman subsystem but at level 5 (notice). This is why logging the netman subsystem at level 5 will show all state transitions. Unless you really want to see everything on each subsystem then you can of course log everything! :)

Unfortunately, I cannot tell you were the CSS is forwarding the syslog messages too. The location of the syslog messages sent by the CSS to host 10.9.x.x is dependent on how you have the syslog application setup on the server. I can tell the logging configuration on your CSS is correct. The load balancer shows it has sent 278 log messages to the 10.9.x.x server. If anything you may want to increase the facility level on your configuration.

Hope this info helps!

- Jason

jkriss360 · ‎09-12-2009

Thanks again Jason,

One more quick question....

I cannot seem to locate a way to set the facility at all. It just seems to be at 1.

Can you send me to the right place?

Thanks again

Jeff

jkriss360 · ‎09-12-2009

I found what I was looking for.

Thanks again.

Gilles Dufour · ‎09-13-2009

Activating the specific logging does not turn on the associated debugging.

You need to know the command to associate the debugging.

But, this is quite dangerous in a live enrionment as you could potentially kill the box with too many messages.

A loadbalancer is not a router, and troubleshooting is not done by activating debug command.

Instead, you need to have a sniffer tool handy so you can start sniffing the traffic and see what is going on.

With some usefull show commands like 'show summary' or 'show service summary' you can 99% of the time solve your problems.

Gilles.