How can I preserve Client IP address?

Unanswered Question
Mar 9th, 2012

I am configuring the ACE for bridged mode. However, the real server is seeing VIP IP but not Client IPs. Our business requires that the real server must see client IPs. Do you have any idea how to set that up?

I tried to turn ON/OFF normalization but it is still not working.

Thanks,

Vincent

==============================

Here is my configuration:

rserver host 192.168.71.71

  ip address 192.168.71.71

  inservice

serverfarm host WEB_FARM

  failaction purge

  probe ICMP

  rserver 192.168.71.71

    inservice

access-list PERMIT-BPDU ethertype permit bpdu

access-list ALL line 8 extended permit ip any any

sticky ip-netmask 255.255.255.255 address source WEB_FARM_Sticky

  timeout 180

  replicate sticky

  serverfarm WEB_FARM

class-map match-all WEB_FARM_VIP

  2 match virtual-address 192.168.71.154 tcp eq 80

class-map type management match-any remote_access

  2 match protocol xml-https any

  4 match protocol icmp any

  5 match protocol telnet any

  6 match protocol ssh any

  7 match protocol http any

  8 match protocol https any

  9 match protocol snmp any

policy-map type loadbalance first-match WEB_FARM_Policy

  class class-default

    sticky-serverfarm WEB_FARM_Sticky

policy-map multi-match WEB_VIPS

  class WEB_FARM_VIP

    loadbalance vip inservice

    loadbalance policy WEB_FARM_Policy

    loadbalance vip icmp-reply active

    nat dynamic 6 vlan 31

    nat dynamic 5 vlan 21

interface vlan 21

  description Client VLAN

  bridge-group 171

  no normalization

  mac-sticky enable

  access-group input PERMIT-BPDU

  access-group input ALL

  service-policy input WEB_VIPS

  nat-pool 5 192.168.71.154 192.168.71.154 netmask 255.255.255.255 pat

interface vlan 31

  description Server VLAN

  bridge-group 171

  no normalization

  mac-sticky enable

  access-group input PERMIT-BPDU

  access-group input ALL

  service-policy input WEB_VIPS

  nat-pool 6 192.168.71.154 192.168.71.154 netmask 255.255.255.255 pat

  no shutdown

interface bvi 171

  ip address 192.168.71.3 255.255.255.0

  no shutdown

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 0 (0 ratings)
DavidZC12 Fri, 03/09/2012 - 17:45

The reason you aren't seeing the client IPs is because of these NAT statements.

interface vlan 21

description Client VLAN

bridge-group 171

no normalization

mac-sticky enable

access-group input PERMIT-BPDU

access-group input ALL

service-policy input WEB_VIPS

nat-pool 5 192.168.71.154 192.168.71.154 netmask 255.255.255.255 pat

interface vlan 31

description Server VLAN

bridge-group 171

no normalization

mac-sticky enable

access-group input PERMIT-BPDU

access-group input ALL

service-policy input WEB_VIPS

nat-pool 6 192.168.71.154 192.168.71.154 netmask 255.255.255.255 pat

no shutdown

This is telling the ACE to Source NAT to these addresses.  You shouldn't need either of these unless you need the source NATing for your rservers to connect outbound to other servers from the VIP address.  At minimum you should be able to remove the NAT statement from the vlan 31 interface.

vtnguyenaz Fri, 03/09/2012 - 18:11

If I removed the nat-pool statements <nat-pool 6 192.168.71.154 192.168.71.154 netmask 255.255.255.255 pat >, the service is not working right away. Any idea?

DavidZC12 Fri, 03/09/2012 - 22:08

Do you have a default route on the ACE and the rservers? Are they all pointing to the same IP? I have the same configuration.  An ACE 4710 in transparent mode, but I have no NATing and my rservers are able to see the original client IPs (security requirement).

Here is part of my config for one serverfarm

rserver host RS_MIDTIER_220
  description
  ip address 172.31.0.131
  inservice
rserver host RS_MIDTIER_221
  description
  ip address 172.31.0.132
  inservice
rserver host RS_MIDTIER_222
  description
  ip address 172.31.0.133
  inservice

rserver redirect RS_SSL_Redirects

  webhost-redirection https://%h/%p 301

  inservice

!
action-list type modify http SSL_URL_REWRITE
  ssl url rewrite location ".*"
!

serverfarm redirect SF_SSL_Redirects

  predictor leastconns

  rserver RS_SSL_Redirects

  inservice

serverfarm host SF_Midtier_Prod
  description Midtier Production
  predictor leastconns
  probe APACHE
  probe ICMP
  rserver RS_MIDTIER_220 80
    inservice
  rserver RS_MIDTIER_221 80
    inservice
  rserver RS_MIDTIER_222 80
    inservice
!
ssl-proxy service SSL_PSERVICE_MIDTIER_PROD
  key
  cert
  chaingroup EntrustChainGroup
!
sticky http-cookie JSESSIONID Sticky_Jsession_Cookie_Midtier_Prod
  timeout 90
  serverfarm SF_Midtier_Prod
!
class-map type management match-any REMOTE_MGT_ACCESS
  description remote access traffic match
  2 match protocol ssh source-address
  4 match protocol https source-address
  5 match protocol snmp source-address
!
class-map match-any VS_Midtier_Prod_L3SLB
  description Midtier Prod IPs
  2 match virtual-address 172.31.0.46 tcp eq https
  3 match virtual-address 172.31.0.47 tcp eq https
!
class-map match-any VS_SSL_Redirects
  description Redirects any http VIPS to https
  5 match virtual-address 172.31.0.46 tcp eq www
  6 match virtual-address 172.31.0.47 tcp eq www
!
policy-map type management first-match REMOTE_MGMT_ALLOW_POLICY
  class REMOTE_MGT_ACCESS
    permit
!
policy-map type loadbalance http first-match Midtier_Prod_L4SLB
  class class-default
    sticky-serverfarm Sticky_Jsession_Cookie_Midtier_Prod
    action SSL_URL_REWRITE
!
policy-map type loadbalance first-match SSL_Redirect_L4SLB
  class class-default
    serverfarm SF_SSL_Redirects
!
policy-map multi-match Farm_VIPS
  class VS_SSL_Redirects
    loadbalance vip inservice
    loadbalance policy SSL_Redirect_L4SLB
  class VS_Midtier_Prod_L3SLB
    loadbalance vip inservice
    loadbalance policy Midtier_Prod_L4SLB
    loadbalance vip icmp-reply active
    ssl-proxy server SSL_PSERVICE_MIDTIER_PROD
!
interface vlan 100
  description DMZ ACE frontside
  bridge-group 1
  access-group input BPDUALLOW
  access-group input ALL
  service-policy input REMOTE_MGMT_ALLOW_POLICY
  service-policy input Farm_VIPS
  no shutdown
interface vlan 110
  description DMZ ACE backside
  bridge-group 1
  access-group input BPDUALLOW
  access-group input ALL
  no shutdown
!
interface bvi 1
  ip address 172.31.0.150 255.255.255.0
  no shutdown

rserver redirect RS_SSL_Redirects

  webhost-redirection https://%h/%p

301

  inservice

!

domain

ip route 0.0.0.0 0.0.0.0 172.31.0.1

vtnguyenaz Fri, 03/09/2012 - 22:19

I have the default route 0.0.0.0 0.0.0.0 192.168.1.1 which is management VLAN. I dont have any default route on rservers. Can I have a copy of your configuration? My email is vtnguyen@gmail.com. I am deeply appreciated your help. 

DavidZC12 Fri, 03/09/2012 - 22:20

I think if you add a default route on your rservers they should start working.  With the source NAT in place they never need to go outside the local subnet.  When you remove the source NAT they can't get back to the client because they don't have a default gateway.  The default route on the ACE should be in the same subnet as the BVI interface.

Did you also remove these lines:

policy-map multi-match WEB_VIPS

class WEB_FARM_VIP

loadbalance vip inservice

loadbalance policy WEB_FARM_Policy

loadbalance vip icmp-reply active

nat dynamic 6 vlan 31

nat dynamic 5 vlan 21

vtnguyenaz Fri, 03/09/2012 - 22:58

Yes, I removed those 2 nat dynamic 5 and 6 lines. It is still not working. I am going through my configuration now.

vtnguyenaz Fri, 03/09/2012 - 23:57

For rservers, what is the gateway you using, 172.31.0.1 or 172.31.0.150?

DavidZC12 Sat, 03/10/2012 - 06:24

Yes, the ACE and the rservers are all on the same subnet and therefore are using the same default gateway 172.31.0.1

vtnguyenaz Mon, 03/12/2012 - 12:28

David,

Do you have only < ip route 0.0.0.0 0.0.0.0 172.31.0.1 > in your ACE or you have more than ip route statement for each different subnet if you have servers are running on different VLAN/subnets?

Thanks,

DavidZC12 Mon, 03/12/2012 - 13:02

All my rservers and VIPs are on the same subnet and the default gateway is a firewall so I only have the one route on the ACE.

jsirstin Mon, 03/12/2012 - 13:11

Vincent,

Can you issue the command "show arp" from the ACE context. Pay attention to what vlan the gateway and servers are in. My suspicion is that your servers are on the same vlan as the gateway where they should be in the vlan behind the ACE. The servers default gateway should still be pointing to the FW address but this server needs to be physically behind the ACE not in front of it. If your servers are in the opposite vlan as the gateway then this is not your problem.

Hope this helps

Jim

vtnguyenaz Mon, 03/12/2012 - 13:16

Hi Jim,

I am using Bridged mode that both Client and Server VLAN are on the same subnet. Your suggestion is Routed mode I think.

Thanks,

Vincent

jsirstin Mon, 03/12/2012 - 13:24

Vincent,

the difference between routed and bridge mode is in routed mode The two vlans are in different subnets. In bridge mode both vlans are in the same subnet. If your servers are showing up in the same vlan as the gateway then you are in a one-armed mode not bridged. The ACE by default does server nat. Client to vip entering the ACE client to server ip when leaving. If the reply from the server does not pass back through the ACE it cannot change the server IP back to the vip that the client talked to.

In bridge mode you have the servers on the vlan behind the ACE so the reply has to pass back through the ACE.

Regards

Jim

vtnguyenaz Mon, 03/12/2012 - 13:29

Jim,


Can you tell me David's setup is in bridged mode or one-armed mode? Since I have identical with his setup. My Client VLAN, Server VLAN, BVI and Real Server IP are on the same subnet and all are using default gateway on the ASA firewall.

Thanks,

Vincent

DavidZC12 Mon, 03/12/2012 - 14:01

Vincent, vlan 100 in my setup is the vlan where the default gateway is (front side of ACE).  All my rservers are in vlan 110 which is logically behind the ACE.  Bridged mode uses one subnet and bridges the two different VLANs.  Unless you are using one-arm mode both bridge mode and routed mode require the traffic to flow through the ACE.

jsirstin Mon, 03/12/2012 - 14:09

Forgot to hit reply but it looks like David answered as well.

The config is bridged but depending on what vlan you server is in it makes it bridged or one armed. If your servers are in the same vlan as the gateway try changing the switch port config so the switchport access vlan command is in the vlan that is opposite of the gateway and test. If it is in the same vlan as the gateway then you are using a one armed mode topology.

Your config is definately a bridge mode config but I think your servers are just in the wrong vlan.

vtnguyenaz Mon, 03/12/2012 - 14:26

Hi David and Jim,

Please correct me! Here is my setup details:

Real server is a VMware box. ESX VM is setup with VLAN 21 for 192.168.71.x subnet.

One the ACE, I set Gig1/1;1/2;1/3 as Ether-channel and all VLAN trunk to another switch, and that switch connects to the firewall.

I have firewall ASA that includes VLAN 21 and VLAN 10. VLAN 21 is 192.168.71.x subnet and VLAN 10 is 192.168.1.x. On the ACE, I defined VLAN 10 for management VLAN and I setup a default route 0.0.0.0 0.0.0.0 192.168.1.1. Also, on the ACE, I create VLAN 21 as Client VLAN and VLAN 31 as Server VLAN, howerver, in my ASA, it doesnt have the VLAN 31.

Do I need to add VLAN 31 to the firewall and ESX VM server? Do I miss something there?

Please advice,

Vincent

DavidZC12 Mon, 03/12/2012 - 15:20

I'm assuming that your ASA is the default gateway for vlan 21.  You need to change your default gateway on the ACE to be the ASA address on that subnet.  I would also change your management interface to a different context on the ACE.

Are your rservers all on VLAN 31 and your ASA on VLAN 21? If so, it sounds like you have everything setup right with the exception of sending all traffic out the management interface.  You could put some static routes for specific subnets or IPs on the management VLAN, but I think the better practice is to probably move your management interface to a different context and allocate that VLAN to that context.

csimmons@appros... Tue, 03/13/2012 - 08:58

I had the same issue. This was the solution provided by TAC.

policy-map type loadbalance first-match LB_lc3q-ui#v51a

  class class-default

    sticky-serverfarm ST_lc3q-ui#v51a

    action SSL_rewrite

    insert-http x-forward header-value "%is"

Your application code will need to read the header in order to get the clients real IP.

-Clint

vtnguyenaz Tue, 03/13/2012 - 09:10

Hi Clint,

Do you have any idea for UDP application?

Thanks,

Vincent

csimmons@appros... Tue, 03/13/2012 - 09:25

From my experience you will need to remove the NAT, otherwise the packets source will be of the ACE NAT.

I know removing the NAT is probably not possible with your configuration. Unfortunately your options are slim.

I would get TAC involved for a final solution.

-Clint

Actions

Login or Register to take actions

This Discussion

Posted March 9, 2012 at 4:34 PM
Stats:
Replies:21 Avg. Rating:
Views:1175 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard

Rank Username Points
1 1,551
2 369
3 333
4 228
5 212
Rank Username Points
5