CSS 11503 - load balancer Tomcat web server

Unanswered Question
Apr 14th, 2009

Hello,

I am new to the CSS 11503. I am looking for sample configurations on loadbalancing a pair of Tomcat web server applications on port 8080. Any assistance will be greatly appreciated.

Regards,

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
sachinga.hcl Wed, 04/15/2009 - 17:45

Hello,

I have a CSS 11501 on which I managed to configure a fake SSL on the

device itself.

Behind the CSS, I have 2 Tomcat servers listening on port 8754.

These 2 Tomcat servers do not have any "Web Servers" like Apache or

IIS which would normally listen on port 80.

With my config below, when I hit the Virtual IP of the CSS, httpS://10.136.2.46/,

I get a prompt for a SSL Certificate warning but since I know its a

fake SSL, I say OK (I use IE 7) and proceed to the site. Then I see a

login page (httpS://....../welcome.do) and as soon as I login with my

credentials, I get re-directed to URL http://10.136.2.46/ instead of

keeping httpS://.....

My intent is to have an httpS:// session maintained throughout with

CSS accepting the incoming encrypted message, does the authentication,

decrypts the message to HTTP and forwards the client request to Tomcat

on port 8754.

Once the results are back from Tomcat, CSS should re-encrypt it and

send the results back to the same client via HTTPS.

----------------------------------------------------------------------------------------------------------------------

CSS11501# show running-config

!Generated on 08/23/2007 16:45:45

!Active version: sg0730005

configure

!*************************** GLOBAL ***************************

ssl associate rsakey keyname keyfile

ssl associate cert certfile certfile

ip route 0.0.0.0 0.0.0.0 10.136.2.254 1

!************************** CIRCUIT **************************

circuit VLAN1

ip address 10.136.2.47 255.255.255.0

no redirects

!*********************** SSL PROXY LIST ***********************

ssl-proxy-list ssl_virtual_server

ssl-server 20

ssl-server 20 vip address 10.136.2.46

ssl-server 20 rsacert certfile

ssl-server 20 rsakey keyname

ssl-server 20 cipher rsa-export-with-rc4-40-md5 10.136.2.46 80

weight 5

ssl-server 20 urlrewrite 22 10.136.2.46 sslport 443 clearport 8754

active

!************************** SERVICE **************************

service http_web1_svc

ip address 10.136.2.20

protocol tcp

port 8754

keepalive type http

active

service http_web2_svc

ip address 10.136.2.48

protocol tcp

port 8754

keepalive type http

active

service https_sslvirtualweb_svc

type ssl-accel

slot 2

keepalive type none

add ssl-proxy-list ssl_virtual_server

active

!*************************** OWNER ***************************

owner NON-SSL

content http_rule

sticky-inact-timeout 150

vip address 10.136.2.46

advanced-balance sticky-srcip

balance leastconn

protocol tcp

port 80

add service http_web1_svc

add service http_web2_svc

active

owner SSL

content https_rule

protocol tcp

vip address 10.136.2.46

port 443

add service https_sslvirtualweb_svc

sticky-serverdown-failover sticky-srcip

application ssl

advanced-balance ssl

active

!*************************** GROUP ***************************

group virtual_web_server_pool

add destination service http_web1_svc

add destination service http_web2_svc

vip address 10.136.2.46

active

CSS11501#

contnued in 2nd page below.........

sachinga.hcl Wed, 04/15/2009 - 17:45

continued from page 1....

2nd Example:

Basically, I use SSL termination on CSS's SSL module and it seems no work for the web server using tomcat. The tomcat server insists to use http, not https. (The server always indicate client to query a URL with http head, not a URL with https head.)

Is it possible to change the behavior from CSS? My configuration is:

ssl-proxy-list ssl-slot3

ssl-server 19

ssl-server 19 vip address 10.1.1.19

ssl-server 19 rsakey testRSAkey

ssl-server 19 rsacert testCAcert

ssl-server 19 cipher rsa-with-rc4-128-sha 10.4.98.19 81

service test-backendserver-8080

ip address 10.3.3.29

protocol tcp

port 8080

keepalive type tcp

keepalive port 8080

active

content ssl-test-rule

vip address 10.1.1.19

protocol tcp

port 443

add service ssl-slot3-srv

active

content ssl-test-rule2

vip address 10.2.2.19

protocol tcp

port 8080

add service test-backendserver-8080

balance leastconn

active

My tomcat server is either using absolute path (so link like http://.....) in this case the only thing you can do is modify the server, or the tomcat server is sending a redirect to the client to http://....

The SSL-mod can intercept the redirect and rewrite it to HTTPS.

So I need to add a command 'ssl-server 19 urlrewrite 1 * sslport 443 clearport 8080'

A sniffer trace would tell us why the client keeps using http.

if it is absolute path, can we use backend-ssl configuration to achive L5 load balancing (with stickness)?

I do not see the benefit of doing backend ssl if the frontend is cleartext.

Unless the CSS is on the clientside and you want to go accross the internet to the server.

In this case, yes, backend ssl could be a solution.

But verify first with a sniffer trace what is going on.

Tomcat's native behavior is to generate

absolute URL's on all redirects. And that cannot be changed.

To fix it is to change configuration on tomcat to treat CSS as a proxy server.

Also get lots of cisco css config examples here below mentioned URL:

1. Configuration Examples and TechNotes

http://www.cisco.com/en/US/products/hw/contnetw/ps792/prod_configuration_examples_list.html

2. CSS product support page

http://www.cisco.com/en/US/products/hw/contnetw/ps789/tsd_products_support_eol_series_home.html

3. Design TechNotes

http://www.cisco.com/en/US/products/hw/contnetw/ps789/tsd_products_support_design_technotes_list.html

4. Configuration guides:

http://www.cisco.com/en/US/products/hw/contnetw/ps789/products_installation_and_configuration_guides_list.html

Hope it iwll help some.

Still get in touch for any further discussion.

Best Regards,

Sachin

sachinga.hcl Wed, 04/15/2009 - 17:56

page 3.....

I also have configured for some companies 2 CSS's as load balancers for a bunch of web servers and they work great . When a web daemon on a server crashes, or is shutdown for one reason or another, the CSS removes that service from the pool, which is good, but that server looses connection to the outside world. We can still SSH into the server and push data onto it, but the route to the outside world is dead.

This was the problem since our servers often need to connect to outside resources on startup.

We were currently set up with a 2-Tier network layout where the CSS is connected to our public network (say 198.202.0.0/24) and our servers sit on a private network(192.168.0.0/24) and use the CSS as their gateway.

I tried a number of fixes to rectify the situation, but nothing has worked so far. The servers have another NIC connected directly to the public network, but since the default GW is the CSS, it does not help. I tried changing the default GW on the servers to point to our public GW, but then access through the CSS VIP does not work. Static routes don't do the trick either.

I try to find is there any configuration option that I can change so that the CSS does not kill the GW for a server whose service is down?

Either that, or do you know of a better way to lay out the configuration so we avoid the problem?

Later on going through various reference guides I find some solution as follows:

CSS by default acts as a router but you need to translate the Real server's ip for server originated traffic.

To have the CSS nat traffic initiated by the server side you need to add a "source group" into the CSS config. Here is what you need:

group

vip address

add service

add service

etc...

active

This will force the CSS to NAT server initiated traffic.

We already have groups set up for the servers. I have attached our config file (slightly edited for security reasons) to see our setup.

Here are some more details on our setup.

General:

Public Network: 198.202.0.0/24

Private Network: 192.168.0.0/24

Public Gateway: 198.202.0.13

CSS IP: 198.202.0.1

CSS Gateway: 192.168.0.1

Server Network Setup:

Default Gateway -> CSS Gateway (192.168.0.1)

Eth0 198.202.0.10#

Eth1 192.168.0.10#

As I said, while the Tomcat daemon is running, the server can ping to the outside world through the CSS, but as soon as the daemon is stopped, the keepalive detects it as down, removes it from the pool and drops all outgoing packets from that server.

you try using a different IP (other than 198.202.0.51 )in group configuration.

group webservers

add service http2

add service http3

add service http1

vip address 198.202.0.x

active

....continuue to page 4

sachinga.hcl Wed, 04/15/2009 - 17:59

page 4.....

Complete config after few changes in real ip address seems as follows:

!Generated on 23/11/2007 09:33:21

!Active version: sg0750004

configure

!*************************** GLOBAL ***************************

global-portmap base-port 3000 range 30000

bridge spanning-tree disabled

cdp run

sntp primary-server 132.249.0.88 version 1

username admin des-password xxxxxxxxxxx superuser

no restrict web-mgmt

app

app session 198.202.0.2

logging subsystem syssoft level notice-5

logging subsystem buffer level notice-5

logging subsystem flowmgr level notice-5

logging subsystem radius level notice-5

logging subsystem wcc level notice-5

logging subsystem ipv4 level notice-5

logging subsystem chassis level notice-5

logging subsystem vlanmgr level notice-5

logging subsystem netman level notice-5

logging subsystem app level notice-5

logging subsystem rip level notice-5

logging subsystem ospf level notice-5

logging subsystem sntp level notice-5

logging subsystem dhcp level notice-5

logging subsystem vrrp level notice-5

logging subsystem redundancy level notice-5

logging subsystem csdpeer level notice-5

logging subsystem portmapper level notice-5

logging subsystem circuit level notice-5

logging subsystem security level notice-5

logging subsystem fac level notice-5

logging subsystem vpm level notice-5

logging subsystem publish level notice-5

logging subsystem acl level notice-5

logging subsystem keepalive level notice-5

logging subsystem urql level notice-5

logging subsystem nql level notice-5

logging subsystem dql level notice-5

logging subsystem pcm level notice-5

logging subsystem proximity level notice-5

logging subsystem hfg level notice-5

logging subsystem replicate level notice-5

logging subsystem boomerang level notice-5

logging subsystem fp-driver level notice-5

logging subsystem flowagent level notice-5

logging subsystem cdp level notice-5

logging subsystem asr level notice-5

logging subsystem natmgr level notice-5

logging subsystem ssl-accel level notice-5

logging subsystem perform level notice-5

logging subsystem sshd level notice-5

logging subsystem reporter level notice-5

dns suffix sdsc.edu

dns primary 198.202.0.26

dns secondary 132.249.0.26

ip route 0.0.0.0 0.0.0.0 198.202.0.13 1

!************************* INTERFACE *************************

interface 1/1

description "front-side network"

bridge vlan 522

interface Ethernet-Mgmt

phy 10Mbits-FD

interface 2/1

description "back-end network"

bridge vlan 525

!************************** CIRCUIT **************************

circuit VLAN522

ip address 198.202.0.1 255.255.255.0

circuit VLAN525

ip address 192.168.0.1 255.255.255.0

!************************* KEEPALIVE *************************

keepalive get-homepage1

description "HTTP GET Keepalive for HTTP1"

ip address 192.168.0.101

method get

frequency 15

retryperiod 15

type http

uri "/pdb/cssKeepAlive.do"

active

keepalive get-homepage2

ip address 192.168.0.102

description "HTTP GET Keepalive for HTTP2"

type http

method get

frequency 15

retryperiod 15

uri "/pdb/cssKeepAlive.do"

active

keepalive get-homepage3

ip address 192.168.0.103

description "HTTP GET Keepalive for HTTP3"

type http

method get

frequency 15

retryperiod 15

uri "/pdb/cssKeepAlive.do"

active

......continue to page 5

sachinga.hcl Wed, 04/15/2009 - 17:59

page 5.....

!************************** SERVICE **************************

service http1

ip address 192.168.0.101

port 80

keepalive type named get-homepage1

keepalive port 80

active

service http2

ip address 192.168.0.102

keepalive port 80

keepalive type named get-homepage2

port 80

active

service http3

keepalive port 80

ip address 192.168.0.103

keepalive type named get-homepage3

port 80

active

!*************************** OWNER ***************************

owner pdb

content pdb-http

protocol tcp

port 80

advanced-balance sticky-srcip-dstport

add service http2

add service http3

add service http1

vip address 198.202.0.51

active

!*************************** GROUP ***************************

group webservers

add service http2

add service http3

add service http1

vip address 198.202.0.51

active

Hope it will be of some help to you..

Sachin

kin.mak Thu, 04/16/2009 - 03:50

Thank you very much Sachin. The configurations you provided are very helpful.

Actions

This Discussion