CSS Access to VIP not working, access to real server IP working.

dtochilovsky · ‎10-10-2008

I have an issue with access to a VIP on the CSS. The CSS is designed as a Dual Arm with two interface bridging a L2 L/B Vlan and a L2 Non L/B Vlan. The firewall in front of the CSS acts as a L3 gateway.

Client > FW (L3 VLAN) > L2 Switch (L2) > CSS > L/B Server

I can get to the real server IP and port combination fine but hitting the VIP I see the flow created on the CSS and the return flow is sent but the firewall in front of the CSS never sees the SYN/ACK packet from the server.

Here are the flows for traffic :

sh flows 192.168.4.6

--------------- ----- --------------- ----- --------------- --- ------- ------

Src Address SPort Dst Address DPort NAT Dst Address Prt InPort OutPort

--------------- ----- --------------- ----- --------------- --- ------- ------

192.168.4.6 18397 10.65.16.40 8080 10.65.16.62 TCP e1 e2

sh flows 10.65.16.62

--------------- ----- --------------- ----- --------------- --- ------- ------

Src Address SPort Dst Address DPort NAT Dst Address Prt InPort OutPort

--------------- ----- --------------- ----- --------------- --- ------- ------

10.65.16.62 8080 192.168.4.6 18397 192.168.4.6 TCP e2 e1

I sniffed the traffic on the server and it replis to the original SYN packet and there's only one interface of that server.

What is strange is that I can get to the VIP successfully from the servers that are bridged to the CSS, like the non-load balanced servers in the same L3 VLAN. So traffic that does not need to pass through the firewall seems to work fine.

Sounds like a symetrical routing issues but I can't find the problem.

Any help would be greately appreciated.

Thank you.

Dmitry.

dtochilovsky · ‎10-13-2008

Correction to the above, the flow should be :

Client > FW (L3 VLAN) > L2 Switch (L2) > CSS > L2 Switch (L2) > L/B Server

Dmitry.

Syed Iftekhar Ahmed · ‎10-13-2008

What is the default gateway on the server?

Is it the Circuit IP on CSS?

Is there any possibility that the response from the server is bypassing CSS?

Syed

dtochilovsky · ‎10-13-2008

Hi Syed.

The Server's default gateway is the FW L2 interface. (10.65.16.1 configured on the FW)

The Circuit IP address of the CSS is different - 10.65.16.8.

I do not think the server's reply is bypassing the CSS as I see the following flow

on the CSS from the server destined to the client :

sh flows 10.65.16.62

--------------- ----- --------------- ----- --------------- --- ------- ------

Src Address SPort Dst Address DPort NAT Dst Address Prt InPort OutPort

--------------- ----- --------------- ----- --------------- --- ------- ------

10.65.16.62 8080 192.168.4.6 18397 192.168.4.6 TCP e2 e1

dmitry.

dtochilovsky · ‎10-14-2008

These are the configurations on the L2 Switch:

New L/B L2 VLAN:

vlan 411

name Prod_Load_Balanced

Existing vlan:

vlan 111

name Prod_Non_Load_Balanced

I bridge the VLAN's to the CSS:

interface FastEthernet4/1

switchport access vlan 111

switchport mode access

speed 100

duplex full

no snmp trap link-status

interface FastEthernet4/2

switchport access vlan 411

switchport mode access

speed 100

duplex full

no snmp trap link-status

These are the interface and circuit configs on the CSS:

interface e1

phy 100Mbits-FD

redundancy-phy

description "Non-Load Balanced"

bridge vlan 111

interface e2

phy 100Mbits-FD

redundancy-phy

description "Load Balanced"

bridge vlan 111

circuit VLAN111

redundancy

ip address 10.65.16.8 255.255.255.0

Here is the service and content config from the CSS:

service apefl03d

protocol tcp

ip address 10.65.16.62

keepalive port 8080

port 8080

keepalive type script ap-kal-tcp-ports "10.65.16.62 8080"

keepalive frequency 30

active

owner APP

content Production

add service apefl03d

protocol tcp

port 8080

vip address 10.65.16.40

active

The Server's default gateway is the FW interface, 10.65.16.1.

dmitry.

dtochilovsky · ‎10-30-2008

This issue has been resolved. The firewall has two "inside" interfaces and the CSS was using the wrong MAC address (the second FW interface) to send the reply traffic.

I had to remove the "ip uncond-bridging" command and re-apply it to get the traffic to flow properly and also had to set up static routes to point to the default gateway of the FW for the particular subnet.

My understanding was that "ip uncond-bridging" command takes care of bridging and the CSS does not need to use the routing table. But if I remove the static routes traffic does not flow properly, it is being sent to the second FW "inside" interface but traffic comes in on the first "inside" interface and thus the FW silently drops it.