Connections dropping in Bridge Mode

new_networker · ‎10-26-2008

Hi,

I have configured ACE in bridged mode. Connection to VIP from the client to the server VLAN is happening. But any connection to VIP from the server itself in the same server VLAN is failing as CON_CLOSE. Is SNAT required in bridged mode.

Please assist.

SHOW SERVICE-POLICY DETAIL

Service-policy: LB-VIP-FROM-SERVER

class: VIP-ACE-XXX-HTTP

VIP Address: Port:

172.18.90.200 eq 80

loadbalance:

L7 loadbalance policy: LB-ACE-XXX

VIP Route Metric : 77

VIP Route Advertise : DISABLED

VIP ICMP Reply : DISABLED

VIP State: INSERVICE

curr conns : 0 , hit count : 25

dropped conns : 25

client pkt count : 39 , client byte count: 1840

server pkt count : 0 , server byte count: 0

L7 Loadbalance policy : LB-ACE-XXX

class/match : class-default

LB action :

sticky group: ACE-XXX-STICKY

primary serverfarm: ACE-XXX

state: UP

hit count : 33

dropped conns : 0

Below show capture

0018: msg_type: PKT_XMT con_id: 1375731816 other_con_id: 0

0019: msg_type: CON_CLOSE con_id: 301989991 reason: 0

0020: msg_type: CON_CLOSE con_id: 1375731816 reason: 0

0020: msg_type: CON_CLOSE

con_id: 1375731816 reason: 0

src_addr: 172.16.10.99 src_port: 2674

dst_addr: 64.91.231.55 dst_port: 110

l3_protocol: 0 l4_protocol: 6

message_hex_dump:

0x0000: 0000 0102 5200 0068 0000 0000 0000 0000 ....R..h........

0x0010: ac10 0a63 405b e737 0629 0002 0a72 006e ...c@[.7.)...r.n

0x0020: 68af f25e 051b 1400 0564 0180 1200 0067 h..^.....d.....g

0x0030: 0000 0000 0018 0480 2445 0000 0000 0008 ........$E......

0x0040: 0000 017f ffae 004e c056 8f8c f5d8 65a5 .......N.V....e.

0x0050: c056 8f8c 0a89 38a5 0000 0000 0000 0119 .V....8.........

0x0060: 0000 0000 0a89 317f 0000 0000 0000 0000 ......1.........

0x0070: 405b e737 ac10 0a63 0621 0001 006e 0a72 @[.7...c.!...n.r

0x0080: 9750 0da2 0536 1400 05b4 0180 5200 0068 .P...6......R..h

0x0090: 0000 0000 0018 0480 2445 0000 0000 0009 ........$E......

0x00a0: 0000 01bd 16d0 004f 8d28 7347 57a6 9d2e .......O.(sGW...

0x00b0: 8d28 7347 0a89 39ca 0000 0000 0000 0119 .(sG..9.........

0x00c0: 0000 0000 0a89 317f 0000 0000 ......1.....

Syed Iftekhar Ahmed · ‎10-26-2008

You need to

1. Source nat the Server initiated traffic

2. & Apply a service policy on Server side vlan

Something in line of...

policy-map type loadbalance first-match App1-policy

class class-default

serverfarm APP1-SF

policy-map multi-match VLAN10-VIPs

class VIP-App1

loadbalance vip inservice

loadbalance policy App1-policy

loadbalance vip icmp-reply

policy-map multi-match Server2VIP

class VIP-App1

loadbalance vip inservice

loadbalance policy App1-policy

loadbalance vip icmp-reply

nat dynamic 10 vlan 20

interface vlan 10

bridge-group 10

access-group input all-traffice

service-policy input MGMT

service-policy input VLAN10-VIPS <-- policy for all VIPs

no shutdown

interface vlan 20

bridge-group 10

access-group input all-traffice

nat-pool 10 10.10.10.199 10.10.10.199 netmask 255.255.255.0 pat

service-policy input MGMT

service-policy input Server2VIP <- Policy for Servers hitting VIP

no shutdown

For details why you need these modifications. Please read my previous post at

http://forum.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Data%20Center&topic=Application%20Networking&topicID=.ee7814f&fromOutline=&CommCmd=MB%3Fcmd%3Ddisplay_location%26location%3D.2cc20917

Syed Iftekhar Ahmed

new_networker · ‎10-26-2008

Thanks.

What is loadbalance vip icmp-reply used for.

Other than the above, I am experiencing an issue with SSL. I have configured ssl-proxy on the policy-map for VIP eq 443 on server side vlan 'only'. However, I can see that from the servers in the Server vlan, accessing the VIP URL via http (80) prompts for the SSL certificate.

Can this happen ? Can a misconfiguration on ACE cause this (may be multi-match policy for server side vlan) ? Or even a misconfiguration on the server. The SSL is terminated on the ACE.

Regards.

Syed Iftekhar Ahmed · ‎10-26-2008

Vip icmp-reply active enables VIPs to responds back to icmp requests destined towards VIP IP.

Why do you have ssl-proxy defined on Server-side vlan?

(Without server2vip traffic requirement) following should be the sample config.

class-map match-all HTTPS-APP1

match virtual-address 10.10.10.10 tcp eq https

policy-map type loadbalance first-match APP1-POLICy

class class-default

serverfarm APP1-SERVERS

policy-map multi-match VLAN10-VIPS

class HTTPS-VIP

loadbalance vip inservice

loadbalance policy APP1-POLICy

loadbalance vip icmp-reply active

ssl-proxy server APP1-SSL-SERVICE

interface vlan 10 <-- client Vlan

bridge-group 10

access-group input all-traffice

service-policy input MGMT

service-policy input VLAN10-VIPS

no shutdown

interface vlan 20 <-- Server Vlan

bridge-group 10

access-group input all-traffice

service-policy input MGMT

no shutdown

1. In above config "APP1-SSL-SERVICE" will decrypt the traffic.

2. Decrypted traffic will be loadbalanced by "APP1-POLICY"

If your class-map is only filtering HTTPS traffic then SSL service should be applied on port 80 traffic.

Do you have a port 80 rule/VIP for the same app?

If yes then hit the server directly on port 80 and make sure that it is not misconfigured and is not throwing the cert.

Syed Iftekhar Ahmed

new_networker · ‎10-27-2008

The application team requires https on server vlan as well.

I have checked my configs. Hitting port 80 on VIP prompts with the cert. Strange..... Please see below. Is ever it possible that the application server somehow replies to the client to resend the request on port 443 instead of 80 which throws up the cert from ACE. Other than that it is difficult to understand how can this happen.

The result of : http://www.xxx.com (cert is thrown up)

class: VIP-ACE-XXX-HTTP

VIP Address: Port:

10.0.0.200 eq 80

loadbalance:

L7 loadbalance policy: LB-ACE-XXX

VIP Route Metric : 77

VIP Route Advertise : DISABLED

VIP ICMP Reply : ENABLED

VIP State: INSERVICE

curr conns : 0 , hit count : 1

dropped conns : 0

client pkt count : 7 , client byte count: 1361

server pkt count : 6 , server byte count: 1464

L7 Loadbalance policy : LB-ACE-XXX

class/match : class-default

LB action :

sticky group: ACE-XXX-STICKY

primary serverfarm: ACE-XXX

state: UP

hit count : 4

dropped conns : 0

Class-map rule:

class-map match-all VIP-ACE-XXX-HTTP

2 match virtual-address 10.0.0.200 tcp eq www

policy-map multi-match LB-VIP-SERVER-VLAN

class VIP-ACE-XXX-HTTP

loadbalance vip inservice

loadbalance policy LB-ACE-XXX

loadbalance vip icmp-reply

class VIP-ACE-XXX-HTTPS

loadbalance vip inservice

loadbalance policy LB-ACE-XXX

loadbalance vip icmp-reply

ssl-proxy server testssl_proxy

class REAL_SERVERS

nat dynamic 1 vlan 99

interface vlan 99 <--- Server VLAN

bridge-group 1

access-group input PERMIT-VLAN-TRAFFIC

nat-pool 1 10.0.0.120 10.0.0.120 netmask 255.255.255.0 pat

service-policy input permitall

service-policy input LB-VIP-SERVER-VLAN

no shutdown

Please assist.

new_networker · ‎10-27-2008

Ok.

One more thing. When I access the rserver locally (i.e. http://www.xyz.com:7777/test , it still prompts for the certificate. (Note: the VIP was on port 80 which translates to 7777 on the rserver).

It means that the local rserver some how gets in touch with the ACE for the cert or has it locally stored. Head spinning...

Syed Iftekhar Ahmed · ‎10-27-2008

If I understand correctly then you hit your real server directly (bypassing vip) on port 7777 and you got the cert.

Ace has nothing to do with it (As you are not hitting VIP, ACE is simply routing to/from the real server). It means you rserver is configured with SSL certificate.

What is the objective?

As per your configs you are doing SSL offload but if your server is doing SSL then you need to do end2end ssl.

Syed Iftekhar Ahmed