This discussion is locked

Ask the Expert: Understanding and Troubleshooting ACE Loadbalancer

Unanswered Question
Aug 3rd, 2012

Read the bioWith Sivakumar Sukumar


Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions about configuration and troubleshooting the Cisco Application Control Engine (ACE) loadbalancer with Sivakumar Sukumar. The Cisco ACE Application Control Engine Module for Cisco Catalyst 6500 Series Switches and Cisco 7600 Series Routers is a next-generation load-balancing and application-delivery solution. A member of the Cisco family of Data Center 3.0 solutions, the module:

  • Helps ensure business continuity by increasing application availability
  • Improves business productivity by accelerating application and server performance
  • Reduces data center power, space, and cooling needs through a virtualized architecture
  • Helps lower operational costs associated with application provisioning and scaling

Sivakumar Sukumar is an experienced support engineer with the High Touch Technical Support content team, covering all Cisco content delivery network technologies including Cisco Application Control Engine (ACE), Cisco Wide Area Application Services (WAAS), Cisco Content Switching Module, Cisco Content Services Switches, and other content products. He has been with Cisco for more than 2 years, working with major customers to help resolve their issues related to content products. He holds CCNP and DCASI certification.

Remember to use the rating system to let Sivakumar know if you have received an adequate response.

Sivakumar might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Data Center sub-community discussion forum shortly after the event. This event lasts through August 24, 2012. Visit this forum often to view responses to your questions and the questions of other community members.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4.5 (10 ratings)
ajay chauhan Wed, 08/08/2012 - 07:09

Hi,

what are  best steps/commands to look at ACE confguration in regards to troubleshoot issues related to load balancing.

For example - if I know the VIP IP and need to know how many real servers/policies are associated with it just to check things configured for one specific VIP.

Thanks

Ajay

sivaksiv Wed, 08/08/2012 - 10:06

Hi Ajay,

There is a handy CLI command that will pull out relevant ACE running-config for a class-map. It looks like it is available on some 3.x version and on 4.x versions and above. In some code versions it is hidden.

show running-config filter [policy-map name] [class-map name]

This will parse the running config for the configuration that is applicable to the policy-map and class-map that is specified.

Regards,

Siva

bcsisupportlab Mon, 08/13/2012 - 09:51

Hi Siva,

I have a Cisco 6500 and Cisco ACE 4710 with the following configuration/connection.

Cisco 6500's 6/29 (vlan 694 for mgmt) connects to ACE 4710's 1/1

Cisco 6500's 6/30 (vlan 697 for client-side) connects to ACE 4710's 1/2

Cisco 6500's 6/31 (vlan 698 for server-side) connects to ACE 4710's 1/3

***********************Cisco 6500***********************

interface GigabitEthernet6/29

description ACE4710 (Mgmt/Int 1/1)

switchport

switchport access vlan 694

no ip address

no cdp enable

!

interface GigabitEthernet6/30

description ACE4710 (Int 1/2)

switchport

switchport access vlan 697

no ip address

no cdp enable

!

interface GigabitEthernet6/31

description ACE4710 (Int 1/3)

switchport

switchport access vlan 698

no ip address

no cdp enable

!

interface Vlan694

ip address 10.78.2.1 255.255.255.248

interface Vlan697

ip address 10.10.40.1 255.255.255.0

!

interface Vlan698

ip address 10.10.50.1 255.255.255.0

***********************ACE 4710***********************

ACE4710/Admin# show run

Generating configuration....

boot system image:c4710ace-mz.A3_2_1.bin

boot system image:c4710ace-mz.A1_8_0a.bin

hostname ACE4710

interface gigabitEthernet 1/1

   switchport access vlan 694

   no shutdown

interface gigabitEthernet 1/2

   description Client-side

   switchport access vlan 697

   no shutdown

interface gigabitEthernet 1/3

   description Server-side

   switchport access vlan 698

   no shutdown

interface gigabitEthernet 1/4

   shutdown

access-list ALL line 8 extended permit ip any any

class-map type management match-any remote_access

   2 match protocol xml-https any

   3 match protocol icmp any

   4 match protocol telnet any

   5 match protocol ssh any

   6 match protocol http any

   7 match protocol https any

   8 match protocol snmp any

policy-map type management first-match remote_mgmt_allow_policy

   class remote_access

     permit

interface vlan 694

   ip address 10.78.2.2 255.255.255.248

   access-group input ALL

   service-policy input remote_mgmt_allow_policy

   no shutdown

interface vlan 697

   ip address 10.10.40.2 255.255.255.0

   fragment chain 112

   access-group input ALL

   no shutdown

interface vlan 698

   ip address 10.10.50.2 255.255.255.0

   fragment chain 112

   access-group input ALL

   no shutdown

ip route 0.0.0.0 0.0.0.0 10.78.2.1

I will assign other ports with the assicated vlans for client and server on the Cisco 6500. Is it a valid setup/configuration?

If not, what should I change? How to make sure that the client traffic and server traffic can be handled by the ACE 4710? Any suggestion for configuration?

Thanks a lot.

Philip

sivaksiv Mon, 08/13/2012 - 11:02

Hi Philip,

Thanks for your question.

The configuration looks good for basic management setup.

Attached the configuration for client to server communication via ACE.

rserver host SERVER_01
  ip address 10.10.50.x
  inservice
rserver host SERVER_02
  ip address 10.10.50.x
  inservice

serverfarm host REAL_SERVERS
  rserver SERVER_01
    inservice
  rserver SERVER_02
    inservice

class-map match-all VIP
  2 match virtual-address 10.10.40.x any

policy-map type loadbalance first-match SLB_LOGIC
  class class-default
    serverfarm REAL_SERVERS
policy-map multi-match CLIENT_VIPS
  class VIP
    loadbalance vip inservice
    loadbalance policy SLB_LOGIC
    loadbalance vip icmp-reply active

interface vlan 697
  service-policy input CLIENT_VIPS
  no shutdown

Also here is a guide to setup basic server loadbalancing with step by step configuration.

http://docwiki.cisco.com/wiki/Cisco_ACE_4700_Series_Appliance_Quick_Start_Guide,_Release_A3(1.0)_--_Configuring_Server_Load_Balancing

Let me know if you have any questions.

Regards,

Siva

bcsisupportlab Mon, 08/13/2012 - 16:07

First, thank you for the quick note, Siva.

However, I am not able to browse/ping the VIP web server from client now.

VIP: 10.10.40.2

Real Web Server IP: 10.10.50.3

Client IP on vlan 697: 10.10.40.3

Here is what I have now.

***********************ACE 4710***********************

hostname ACE4710

interface gigabitEthernet 1/1

  switchport access vlan 694

  no shutdown

interface gigabitEthernet 1/2

  description Client-side

  switchport access vlan 697

  no shutdown

interface gigabitEthernet 1/3

  description Server-side

  switchport access vlan 698

  no shutdown

interface gigabitEthernet 1/4

  shutdown

access-list ALL line 8 extended permit ip any any

probe http 1

  interval 15

  passdetect interval 60

  request method get url http://10.10.50.3

  open 10

rserver host SERVER_01

  ip address 10.10.50.3

  conn-limit max 4000000 min 4000000

  inservice

rserver host SERVER_02

  ip address 10.10.50.4

  conn-limit max 4000000 min 4000000

  inservice

serverfarm host REAL_SERVERS

  probe 1

  rserver SERVER_01 80

    conn-limit max 4000000 min 4000000

    inservice

  rserver SERVER_02 80

    conn-limit max 4000000 min 4000000

    inservice

class-map match-all VIP

  2 match virtual-address 10.10.40.20 any

class-map match-all VIP2

  2 match virtual-address 10.10.40.20 tcp eq www

class-map type management match-any remote_access

  2 match protocol xml-https any

  3 match protocol icmp any

  4 match protocol telnet any

  5 match protocol ssh any

  6 match protocol http any

  7 match protocol https any

  8 match protocol snmp any

policy-map type management first-match remote_mgmt_allow_policy

  class remote_access

    permit

policy-map type loadbalance first-match SLB_LOGIC

  class class-default

    serverfarm REAL_SERVERS

policy-map type loadbalance first-match VIP2-l7slb

  class class-default

    serverfarm REAL_SERVERS

policy-map multi-match CLIENT_VIPS

  class VIP

    loadbalance vip inservice

    loadbalance policy SLB_LOGIC

    loadbalance vip icmp-reply active

  class VIP2

    loadbalance vip inservice

    loadbalance policy VIP2-l7slb

interface vlan 694

  ip address 10.78.2.2 255.255.255.248

  access-group input ALL

  service-policy input remote_mgmt_allow_policy

  no shutdown

interface vlan 697

  ip address 10.10.40.2 255.255.255.0

  fragment chain 112

  access-group input ALL

  service-policy input CLIENT_VIPS

  no shutdown

interface vlan 698

  ip address 10.10.50.2 255.255.255.0

  fragment chain 112

  access-group input ALL

  no shutdown

ip route 0.0.0.0 0.0.0.0 10.78.2.1

Thanks.
Philip

sivaksiv Mon, 08/13/2012 - 22:32

Hi Philip,

Are you able to ping the gateway, servers and 6500 SVI's from the ACE? Can you send me the output of "show service-policy detail" & "show arp"?

Regards,

Siva

bcsisupportlab Tue, 08/14/2012 - 09:56

Hi Siva,

With the above configuration, I can ping server-side's gateway (IP: 10.10.50.1) and client-side's gateway (IP: 10.10.40.1) from ACE. However, I can't ping (IP: 10.10.50.2) and (IP: 10.10.40.2) from 6500. I thought the icmp is allowed from the above configuration.

ACE4710/Admin# show service-policy detail

Policy-map : CLIENT_VIPS

Status     : ACTIVE

Description: -

-----------------------------------------

Interface: vlan 1 697

  service-policy: CLIENT_VIPS

    class: VIP

     VIP Address:    Protocol:  Port:

     10.10.40.20     any

      loadbalance:

        L7 loadbalance policy: SLB_LOGIC

        VIP ICMP Reply       : ENABLED-WHEN-ACTIVE

        VIP state: OUTOFSERVICE

        Persistence Rebalance: DISABLED

        curr conns       : 0         , hit count        : 24       

        dropped conns    : 24       

        client pkt count : 36        , client byte count: 1728               

        server pkt count : 0         , server byte count: 0                  

        conn-rate-limit      : 0         , drop-count : 0        

        bandwidth-rate-limit : 0         , drop-count : 0        

        L7 Loadbalance policy : SLB_LOGIC

          class/match : class-default

            LB action :

               primary serverfarm: REAL_SERVERS

                    state: DOWN

                backup serverfarm : -

            hit count        : 24       

            dropped conns    : 0        

            compression      : off

      compression:

        bytes_in  : 0                  

        bytes_out : 0                  

        Compression ratio : 0.00%

    class: VIP2

     VIP Address:    Protocol:  Port:

     10.10.40.20     tcp        eq    80  

      loadbalance:

        L7 loadbalance policy: VIP2-l7slb

        VIP ICMP Reply       : ENABLED-WHEN-ACTIVE

        VIP state: OUTOFSERVICE

        Persistence Rebalance: DISABLED

        curr conns       : 0         , hit count        : 0        

        dropped conns    : 0        

        client pkt count : 0         , client byte count: 0                  

        server pkt count : 0         , server byte count: 0                  

        conn-rate-limit      : 0         , drop-count : 0        

        bandwidth-rate-limit : 0         , drop-count : 0        

        L7 Loadbalance policy : VIP2-l7slb

          class/match : class-default

            LB action :

               primary serverfarm: REAL_SERVERS

                    state: DOWN

                backup serverfarm : -

            hit count        : 0        

            dropped conns    : 0        

            compression      : off

      compression:

        bytes_in  : 0                  

        bytes_out : 0                  

        Compression ratio : 0.00%

ACE4710/Admin# show arp

Context Admin

================================================================================

IP ADDRESS      MAC-ADDRESS        Interface  Type      Encap  NextArp(s) Status

================================================================================

10.78.2.1       00.12.da.10.3c.0a  vlan694   GATEWAY    5      38 sec       up

10.78.2.2       00.1b.24.3d.bc.8c  vlan694   INTERFACE  LOCAL     _         up

10.10.40.1      00.12.da.10.3c.0a  vlan697   LEARNED    8      10629 sec    up

10.10.40.2      00.1b.24.3d.bc.8c  vlan697   INTERFACE  LOCAL     _         up

10.10.40.3      00.50.56.94.2f.ba  vlan697   LEARNED    7      10589 sec    up

10.10.40.20     00.1b.24.3d.bc.8c  vlan697   VSERVER    LOCAL     _         up

10.10.50.1      00.12.da.10.3c.0a  vlan698   LEARNED    9      4601 sec     up

10.10.50.2      00.1b.24.3d.bc.8c  vlan698   INTERFACE  LOCAL     _         up

10.10.50.3      00.50.56.93.00.db  vlan698   RSERVER    10     235 sec      up

10.10.50.4      00.00.00.00.00.00  vlan698   RSERVER    -       * 1 req     dn

================================================================================

Total arp entries 10

Thanks.
Philip

sivaksiv Tue, 08/14/2012 - 10:10

Hi Philip,

The VIP state is OUTOFSERVICE.

Can you remove the probe from serverfarm and check if the VIP changes to INSERVICE and serverfarm comes UP?

serverfarm host REAL_SERVERS

probe 1      <<<<<<<<<<<<<< REMOVE>>>>>>>>>>>>>>

VIP state: OUTOFSERVICE   <<<<<<<<<<<<<<<

Persistence Rebalance: DISABLED

curr conns : 0 , hit count : 24

dropped conns : 24

client pkt count : 36 , client byte count: 1728

server pkt count : 0 , server byte count: 0

conn-rate-limit : 0 , drop-count : 0

bandwidth-rate-limit : 0 , drop-count : 0

L7 Loadbalance policy : SLB_LOGIC

class/match : class-default

LB action :

primary serverfarm: REAL_SERVERS

state: DOWN  <<<<<<<<<<<<<<<<<<<<

Once VIP is INSERVICE check if you can ping the VIP - 10.10.40.20 from 6500.

If it still shows OUTOFSERVICE after removing the probe send me the output of "show serverfarm detail" & "show rserver detail"

Also can you check if you are able to ping 10.78.2.2 from 6500?

Regards,
Siva

bcsisupportlab Tue, 08/14/2012 - 10:46

Hi Siva,

I can ping VIP (IP: 10.10.40.20) from 6500, also from the client-side (IP: 10.10.40.3), but I still can't browse from the VIP for both webservers when I tried http://10.10.40.20 from the client's web browser. I even see reset from the pcap.

I can browse both webservers direct without an issue though.

ACE4710/Admin# show service-policy detail

Policy-map : CLIENT_VIPS

Status     : ACTIVE

Description: -

-----------------------------------------

Interface: vlan 1 697

  service-policy: CLIENT_VIPS

    class: VIP

     VIP Address:    Protocol:  Port:

     10.10.40.20     any

      loadbalance:

        L7 loadbalance policy: SLB_LOGIC

        VIP ICMP Reply       : ENABLED-WHEN-ACTIVE

        VIP State: INSERVICE

        Persistence Rebalance: DISABLED

        curr conns       : 0         , hit count        : 28       

        dropped conns    : 28       

        client pkt count : 42        , client byte count: 2016               

        server pkt count : 0         , server byte count: 0                  

        conn-rate-limit      : 0         , drop-count : 0        

        bandwidth-rate-limit : 0         , drop-count : 0        

        L7 Loadbalance policy : SLB_LOGIC

          class/match : class-default

            LB action :

               primary serverfarm: REAL_SERVERS

                    state: UP

                backup serverfarm : -

            hit count        : 28       

            dropped conns    : 0        

            compression      : off

      compression:

        bytes_in  : 0                  

        bytes_out : 0                  

        Compression ratio : 0.00%

    class: VIP2

     VIP Address:    Protocol:  Port:

     10.10.40.20     tcp        eq    80  

      loadbalance:

        L7 loadbalance policy: VIP2-l7slb

        VIP ICMP Reply       : ENABLED-WHEN-ACTIVE

        VIP State: INSERVICE

        Persistence Rebalance: DISABLED

        curr conns       : 0         , hit count        : 0        

        dropped conns    : 0        

        client pkt count : 0         , client byte count: 0                  

        server pkt count : 0         , server byte count: 0                  

        conn-rate-limit      : 0         , drop-count : 0        

        bandwidth-rate-limit : 0         , drop-count : 0        

        L7 Loadbalance policy : VIP2-l7slb

          class/match : class-default

            LB action :

               primary serverfarm: REAL_SERVERS

                    state: UP

                backup serverfarm : -

            hit count        : 0        

            dropped conns    : 0        

            compression      : off

      compression:

        bytes_in  : 0                  

        bytes_out : 0                  

        Compression ratio : 0.00%

ACE4710/Admin# show serverfarm detail

serverfarm     : REAL_SERVERS, type: HOST

total rservers : 2

active rservers: 2

description    : -

state          : ACTIVE

predictor      : ROUNDROBIN

failaction     : -

back-inservice    : 0

partial-threshold : 0

num times failover       : 5

num times back inservice : 7

total conn-dropcount : 0

---------------------------------

                                                ----------connections-----------

       real                  weight state        current    total      failures

   ---+---------------------+------+------------+----------+----------+---------

   rserver: SERVER_01

       10.10.50.3:80         8      OPERATIONAL  0          0          18

         max-conns            : 4000000   , out-of-rotation count : 0

         min-conns            : 4000000  

         conn-rate-limit      : -         , out-of-rotation count : -

         bandwidth-rate-limit : -         , out-of-rotation count : -

         retcode out-of-rotation count : -

   rserver: SERVER_02

       10.10.50.4:80         8      OPERATIONAL  0          0          0

         max-conns            : 4000000   , out-of-rotation count : 0

         min-conns            : 4000000  

         conn-rate-limit      : -         , out-of-rotation count : -

         bandwidth-rate-limit : -         , out-of-rotation count : -

         retcode out-of-rotation count : -

ACE4710/Admin# show rserver detail

rserver              : SERVER_01, type: HOST

state                : OPERATIONAL (verified by arp response)

description          : -

max-conns            : 4000000   ,  out-of-rotation count  : 0

min-conns            : 4000000  

conn-rate-limit      : -         ,  out-of-rotation count  : -

bandwidth-rate-limit : -         ,  out-of-rotation count  : -

weight               : 8

---------------------------------

                                                ----------connections-----------

       real                  weight state        current    total              

   ---+---------------------+------+------------+----------+--------------------

   serverfarm: REAL_SERVERS

       10.10.50.3:80         8      OPERATIONAL  0          0                  

         max-conns            : 4000000   ,  out-of-rotation count  : 0

         min-conns            : 4000000  

         conn-rate-limit      : -         ,  out-of-rotation count  : -

         bandwidth-rate-limit : -         ,  out-of-rotation count  : -

         total conn-failures  : 18

rserver              : SERVER_02, type: HOST

state                : OPERATIONAL (verified by arp response)

description          : -

max-conns            : 4000000   ,  out-of-rotation count  : 0

min-conns            : 4000000  

conn-rate-limit      : -         ,  out-of-rotation count  : -

bandwidth-rate-limit : -         ,  out-of-rotation count  : -

weight               : 8

---------------------------------

                                                ----------connections-----------

       real                  weight state        current    total              

   ---+---------------------+------+------------+----------+--------------------

   serverfarm: REAL_SERVERS

       10.10.50.4:80         8      OPERATIONAL  0          0                  

         max-conns            : 4000000   ,  out-of-rotation count  : 0

         min-conns            : 4000000  

         conn-rate-limit      : -         ,  out-of-rotation count  : -

         bandwidth-rate-limit : -         ,  out-of-rotation count  : -

         total conn-failures  : 0

Thanks.
Philip

sivaksiv Tue, 08/14/2012 - 11:11

Hi Philip,

Was this capture taken from client? If so can you apply NAT on ACE and see if it works.

 

policy-map multi-match CLIENT_VIPS

class VIP

loadbalance vip inservice

loadbalance policy SLB_LOGIC

loadbalance vip icmp-reply active

nat dynamic 1 vlan 698     <<<<<<<<<<< ADD  >>>>>>>>>>>.

interface vlan 698

nat-pool 1 10.10.50.10 10.10.50.10 netmask 255.255.255.255 pat   <<<<<<<<<<< ADD >>>>>>>>>>>.

Regards,
Siva

bcsisupportlab Tue, 08/14/2012 - 11:39

Hi Siva,

Yes the pcap is from the client. It works now. Could you explain a little bit of the reason why we need the nat here?

We have the client(IP: 10.10.40.3) sending GET a resquest to ACE's VIP (IP: 10.10.40.20). ACE's VIP goes to real servers (IP: 10.10.50.3, 10.10.50.4). Real servers reply to the nat (IP: 10.10.50.10) and maps it back to the VIP -> client?

Where is the NAT (IP: 10.10.50.10) playing? What's the logic?

Thanks.
Philip

sivaksiv Tue, 08/14/2012 - 11:54

Hi Philip,

That's correct.

The problem was due to asymmetric routing and the server replies directly back to the client bypassing ACE.

The trick here is getting return traffic from the real server to go back through the ACE; this is achieved with source NAT. We create a NAT pool on the ACE and when the user hits the ACE, his address is translated to one in the pool. The real server sees the source address as one in the pool and knows that that subnet resides on the ACE, so server replies to the ACE. The ACE then NATs the address to the user’s real address and forwards the response.

Another option is to change the routing on server so it always responds backs to ACE instead of replying directly back to the client.

Regards,
Siva

bcsisupportlab Tue, 08/14/2012 - 15:04

Thank you for the information, Siva.

So does it mean putting ip route 10.10.50.0 255.255.255.0 10.10.50.2 on the 6500 will take care of it then?

Thanks.
Philip

sivaksiv Tue, 08/14/2012 - 20:05

Hi Philip,

It has to be on the server itself, changing the default gateway on server to ACE ip should work here.

Regards,
Siva

cscherb Wed, 08/15/2012 - 00:30

Is or will ACE loadbalancer be capabel to deal with WebSocket protocoll as defined in RFC 6455 ?

How to deal with stickiness in this area ? My on lab experiments are showing that ip based stickniess is working with ACE software version A4(1.0) - but SessionID based stickiness is not possible.

sivaksiv Wed, 08/15/2012 - 02:33

Hi,

Thanks for your question.

There are no immediate plans to support websocket on ACE and no roadmap available yet. I can tell from previous documented cases and from my personal experience on cases I've handled, there is a particular requirement which seems to be very important for WebSocket traffic.

As WebSocket requires stickiness, to enable all connections from a single user to stick to one server and is particularly effective (and sometimes strictly necessary) when the application requires user authentication, as otherwise,
traffic would be bouncing between two or more servers.

The type of stickiness that you would implement depends entirely on your network requirements.

Since ACE does not have any specific knowledge of the WebSocket protocols, it doesn't have the capability to do deeper protocol inspection but it seem to work for generic Connection based Level 3 and 4 load balancing which I believe you have already tested in your LAB.

You can also get in touch with your cisco internal contact, share the use case and more details to help assist on your requirement.

Regards,

Siva

akhtar.samo Wed, 08/15/2012 - 23:44

Hi Siva,

Good to see the ACE discussion in the Experts Corner. My query is if there is any permanent fix to CSCsz65679 which causes ACE-20 to crash couple of times in a year ? I have noticed that RMA is not a fix for the problem neither the image upgrade. One of our customer had 10's of ACE-20s and neither RMA nor the upgrades fixed the 'NP Control Store Parity Error', so far they have observed around 10 total ACE-20 crashes on different modules in 3 years of time. The upgrades only reduces the crash frequency, probably due to explicit reload during upgrades which refreshes all the buffers.

I believe this might be an issue with the ACE-20 architecture ? similar issues have not been observed on ACE-30.

Regards,

Akhtar

sivaksiv Thu, 08/16/2012 - 01:06

Hi Akhtar,

Thanks for your question.

Sorry your customer had to experience too many crashes due to parity issue.

First let me expalain few things about SRAM. SRAM parity error presented in the core file is not due to a software issue. The issue is the result of a "bit-flip" within the SRAM itself which can occur as a result of environmental conditions. This "bit-flip" is rectified by a simple reboot of the system, which would occur with the generation of the core file.Our testing has shown that these type of issues can occur with very low frequency and if a particular module experiences a significantly higher failure rate and you are running a version which has all the possible workarounds for CSCsz65679 then a proactive RMA could be in order.

ACE20 is susceptible to this because of the way it uses SRAM to store  control information and packet data as  opposed to scratch-pad storage.  Almost any 1-bit flip will be detected as a parity error.

Unfortunately, SRAM's are very sensitive to light, dust, radiation,  shock, temperature,... so it is possible to get an SRAM parity error on  an healthy ACE.

You are right about ACE30, neither ACE4710 or ACE30 are affected by these issues as the design does not use sram  or nitrox.

Also note that we have EOL notice for ACE10/20:

http://www.cisco.com/en/US/prod/collateral/modules/ps2706/end_of_life_c51-674430.html

Regards,

Siva

eng__mohamed Sat, 08/18/2012 - 16:27

Hi Siva

i have two ace module , the standby one is reload sudden , how can i know the cause of this

sivaksiv Sat, 08/18/2012 - 23:16

Hi,

Thanks for your question.

I understand the standby ACE had an unexpected reload, do you see any crash info generated under "dir core:" after reload. If so please send those files to me to determine the reason for reload.

Otherwise you can raise a tac case and attach the following information for our analysis to determine the root cause.

1- 'show tech' on the switch

2- 'show tech' on the Admin context on the ACE

3- Logs on the switch covering the period when the reload happened.

4- Crash files from ACE located under "dire core:"

Let me know if you have any qusetions.

Regards,

Siva

eng__mohamed Sun, 08/19/2012 - 11:09

Hi Siva

I atteched the requied files , but regarding to the crash info , i didnt find crash info for the reload date ( 18 Aug 2012 11:41 PM )

Thanks Siva

Best Regards

Mohamed Abd EL Razik

sivaksiv Sun, 08/19/2012 - 23:56

Hi,

Thanks for providing the data.

This looks like a silent reboot and SUP initiated the reload.

However the information doesn't really explain why it happened. Silent reboots are tricky as they don't leave much data to work with.

Here is the defect that we logged to track the silent reboot. With high probability a SW upgrade will be necessary as few bugs related to silent reloads have been fixed in A2(3.3) and current version is A2(3.5)) and then monitor device.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCsy91540

There is an action plan to determine if this was traffic related L7 or management traffic ANM, XML, SNMP... which may be filling up the resources on ACE that caused the reload.

I can send you the detailed action plan via PM if reqiured.

Let me know if you have any questions.

Regards,
Siva

pipaulo Mon, 08/20/2012 - 00:20

Hi Siva,

Just a note on versions available. We recently appear to have run into the following Bug and had to downgrade to version A2(3.3) as removing our HTTP health probes did not seem like a workable solution for us.

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtz47825

Once downgraded the paired modules stabalised (no longer re-loaded continuously). Both modules were in this state.

Just thought would provide some input.

Thanks.

Paul

sivaksiv Mon, 08/20/2012 - 01:53

Hi Paul,

Thanks for your question.

Its good to know that the devices are stable now after downgrading to A2(3.3) and I am able to track down the TAC case you reported recently on this issue.

Looking into the bug, we had this issue reported mainly on version A2(3.5) in the past and we are working on reproducing the issue on different code versions to find out the reason for memory corruption.

We will have the fix after we successfully reproduce the problem and it has been updated with fixed version as A2(3.7).

Let me know if you have any questions.

Regards,
Siva

eng__mohamed Mon, 08/20/2012 - 03:46

Hi Siva

thanks for the information

kindly send me the detailed action plan to determine if this was traffic related L7 or management traffic ANM, XML, SNMP

Regards,

Mohamed

sivaksiv Mon, 08/20/2012 - 03:57

Hi Mohamed,

Sent you the information via PM. Please check.

Regards,
Siva

eng__mohamed Mon, 08/20/2012 - 04:28

Thanks Siva for your support

Regards

Mohamed

ganessub Mon, 08/20/2012 - 09:01

Hi Siva,

I am attaching the running-config of the ACE which is currently under test in the lab.

As you can see VLAN - 20 is configured to the Client Side & VLAN-30 is configured on the server side.

I am not able to ping the ACE Interface IP address : 2092:dead:beef:cafe::3 from the Cisco Switch ( 7k ) whose interface is connected to the ACE on VLAN-20.

Any idea if this is normal behavior (or) is there any configuration mistake ?

Thanks !!

hostname ACE-4710

interface gigabitEthernet 1/1

  description *** Interface connecting to the UUT-Switch-7k (WS-C7206X) ***

  switchport access vlan 20

  no shutdown

interface gigabitEthernet 1/2

  description *** Interface connecting to the serverfarm ***

  switchport access vlan 30

  no shutdown

interface gigabitEthernet 1/3

  description *** UNUSED***

  no shutdown

interface gigabitEthernet 1/4

  description *** UNUSED***

  no shutdown

access-list everyone  extended permit ip any any

access-list everyone  extended permit pim any any

access-list everyone  extended permit icmp any any

rserver host CNR

ip address 2092:dead:beef:cafe::90

inservice

rserver host CNR-IPv4

ip address 172.27.167.13

inservice

rserver host NMS

ip address 2092:dead:beef:cafe::999

inservice

serverfarm host LABSERVERS

rserver CNR

inservice

rserver CNR-IPv4

inservice

rserver NMS

inservice

! Layer-3 Traffic

class-map type management match-any MGMT

match protocol telnet any

match protocol https any

match protocol http any

match protocol xml-https any

match protocol ssh any

match protocol icmp any

! Layer-4 Traffic

class-map match-all slb-vip-LABSERVERS

match virtual-address 2092:dead:beef:cafe::1 any

! Layer-3 Class-Map defining source traffic. This traffic macthes server initiated

policy-map type management first-match MGMT_POLICY

class MGMT

permit

policy-map type loadbalance first-match LB_POLICY_LABSERVERS

class class-default

serverfarm LABSERVERS

policy-map multi-match CLIENT-VIPS_LABSERVERS

class slb-vip-LABSERVERS

loadbalance vip inservice

loadbalance policy LB_POLICY_LABSERVERS

loadbalance vip icmp-reply active

loadbalance vip advertise active

interface vlan 20

  description "Client Interface"

  bridge-group 1

  access-group input everyone

  service-policy input CLIENT-VIPS_LABSERVERS

  service-policy input MGMT_POLICY

  no shutdown

interface vlan 30

  description "Server Farm"

  bridge-group 1

  service-policy input CLIENT-VIPS_LABSERVERS

  service-policy input MGMT_POLICY

  no shutdown

interface bvi 1

  ipv6 enable

  ip address 2092:dead:beef:cafe::3/64

  description "Client-Server Bridge Group"

  no shutdown

ip route ::/0 2092:dead:beef:cafe::2

username admin password 5 $1$Hh4K/EuN$J9mu8qUJbebWixnC5Wxpo1  role Admin domain

default-domain

username www password 5 $1$9yHPLof8$RZrtAsMV26WtOp/q8Ou8L.  role Admin domain de

fault-domain

*******************************************************************

On the 7200 switch which is connecting to the ACE :

!

interface GigabitEthernet0/3

description Connected to ACE-E1

no ip address

ip pim sparse-mode

ip igmp version 3

ip ospf 1 area 0

shutdown

duplex auto

speed auto

media-type rj45

negotiation auto

ipv6 enable

ipv6 ospf 1 area 0

!

interface GigabitEthernet0/3.20

encapsulation dot1Q 20 native

ipv6 address 2093:DEAD:BEEF:CAFE::2/64

!



ipv6 route 2092:DEAD:BEEF:CAFE::/64 2092:DEAD:BEEF:CAFE::1

*********************************************************************************************************

I am setting it up for a basic management setup & later on progress to enable more functionalities in the ACE.

Please let me know if there are any mistakes (or) corrections which I might need to make in the configuration.

Thanks !

sivaksiv Mon, 08/20/2012 - 11:44

Hi Ganesh,

Thanks for your question.

I dont have a lab setup to test your configuration right now but looking at the config can you verify if the ACE and switch are on the same subnet, looks like the there is a mismatch 2092/2093?

Regards,

Siva

ganessub Mon, 08/20/2012 - 14:40

Siva,

Yes, they are. I am sory about the wrong config on the 6200 switch. It is in the 2092 subnet only.

Please let me know if you can find something wrong.

Thanks a ton !

sivaksiv Mon, 08/20/2012 - 23:38

Hi Ganesh,

If you are using ipv6 then the access-list and the management policy should also be based on ipv6. Or you could first configure using ipv4, test the config and then migrate to ipv6.

sample config below:

access-list everyone  extended permit ip anyv6 anyv6

access-list everyone  extended permit icmpv6 anyv6 anyv6

class-map type management match-any ipv6

  2 match protocol icmpv6 anyv6

Regards,

Siva

fd_case17 Tue, 08/21/2012 - 07:29

HelloSiva,

can i have your action  plan to determine what sort of traffic is causing reboot of our ACE 30 ?

(during upgrade ACE20 > ACE30 process )

i' m suspecting ANM but i' m not sure of that .

thanx in advance KR,

sivaksiv Tue, 08/21/2012 - 08:14

Hi,

Thanks for your question.

The bug I mentioned above is specific to the legacy ACE modules which are the ACE10 and ACE20. It does not apply to the ACE30.

Can you please explain the problem in detail?

Did you see the module reload only at the time of migration? Were you able to complete the migration successfully?

Are there any core files generated under the core directory after reload?

If you could send me any traces/logs I should be able to figure out the reason for reload.

Let me know if you have any questions.

Regards,
Siva

fd_case17 Wed, 08/22/2012 - 09:06

Hello Siva,

here is a log on core:snmpd_log

Service: snmpd

Description: SNMP Agent

Started at Sat Jul 28 17:19:52 2012 (265979 us)

Stopped at Sat Jul 28 17:22:55 2012 (808911 us)

Uptime: 3 minutes 3 seconds

Start type: SRV_OPTION_RESTART_STATELESS (23)

Death reason: SYSMGR_DEATH_REASON_FAILURE_SIGNAL (2)

System image version: A5(1.1) 3.0(0)A5(1.1) adbuild_11:30:52-2011/10/25_/auto/adbure_nightly4/renumber/rel_a5_1_0_throttle/REL_3_0_0_A5_1_1

Image Version : A5(1.1)

System Image Interim version: 3.0(0)A5(1.1) adbuild_11:30:52-2011/10/25_/auto/adbure_nightly4/renumber/rel_a5_1_0_throttle/REL_3_0_0_A5_1_1

Thanx,

did you want more files ?

sivaksiv Wed, 08/22/2012 - 13:44

Hi,

I would require the complete core file off the box for analysis, you could use "copy core: ftp:" to copy the core file to an ftp server.

But looking at the log it looks like the crash occurred on snmpd process.

Can you tell me how often do you see the reload happening?

I came across following defects and they are fixed in A5(2.0) and above. Did you see the problem with A5(2.0)?

CSCua48058 - ACE30 module crash, last boot reason: Service "snmpd"

CSCtx76952 - SNMP crash on ACE30 module

CSCts09006 - ACE crashed on snmpd after receiving SIG4 (SNMP)

Regards,

Siva

ganessub Thu, 08/23/2012 - 08:55

Thanks Siva. The configuration is working . I guess , we missed this command out :

When enabling the ip address for a particular VLAN , this is for the ipv6 configuration which we are currently implementing, I overlooked the " ipv6 enable" command . This ensured that ipv6 routing is established for that vlan.

Thanks for your help !

ferjbello Tue, 08/21/2012 - 08:19

Hi Siva,

Hope you're doing well.

Some weeks ago I had a weird problem with the ACE 4710 appliance.

We had 2 aces working in Fault Tolerance. Behind the ACE appliance, we have 2 Sharepoint 2010 being loadbalanced by the ace. Before the ACE appliance, there is a server that handles the requests from the LAN clients. After receiving those requests, it calls the Sharepoint WEBSERVICES behind the ACE appliance. So, this server, in turn, acts like a client to the call the webservices behind the ACE. The Server behind the ACE and the server (client) before the ACE are in the same VLAN. Since they are in the same VLAN, to get the request go through the ACE appliance, we made a SOURCE NAT. After doing that, every time that the client(server before the ACE) made a request to the Sharepoint servers, we got a 400 bad request error. In order to avoid this problem, we tested the same request from the same client but bypassing the ACE and everything was ok. We did many other tests and found that if we remove the line

"server-conn reuse" from the HTTP parameter PA_HTTP_PERSIS_TCP_REUSE, the error was not happening.

parameter-map type http PA_HTTP_PERSIS_TCP_REUSE

description Http parameter to HTTP Persistance rebalance and TCP Server connection reuse

server-conn reuse  ---------------------- > remove

case-insensitive

persistence-rebalance

set header-maxparse-length 65535

set content-maxparse-length 65535

length-exceed continue

Why could this be happening?

thanks in advance for your help!

Fernando

sivaksiv Tue, 08/21/2012 - 09:13

Hi Fernando,

Thanks for your question.

I'm glad you are able to isolate the problem by removing the conn reuse command off the ACE.

First let me explain how a conn resuse on ACE works:

If a client sends a HTTP/1.1 request  with the "Connection: close" header, to  achieve the back-end connection to be  open even after the  response, ACE will remove the "Connection: close" header and insert the "Connection: Keep-Alive" header in the  request before forwarding it to the rserver.

Now if the client sends a request to the VIP and closes it after receiving the response, then only the client-side connection is removed from the connection database, and the server-side connection is kept in the reuse pool.

Now if the client opens a connection to the VIP and sends a request in it, then this time, instead of opening a new  backend connection to the rserver, ACE  uses the back-end connection in the reuse pool.

So basically TCP server reuse allows the ACE to reduce the number of open connections on a server by allowing connections to persist and be reused by multiple client connections.

The ACE maintains a pool of TCP connections based on TCP options. New client connections can reuse those connections in the pool provided that the new client connections and prior server connections share the same TCP options.

To ensure proper operation of this feature , we need to ensure below :-

  • Ensure that the ACE MSS is the same as the server MSS.
  • Configure port address translation (PAT) on the interface that is connected to the real server.
  • Configure on the ACE the same TCP options that exist on the TCP server.
  • Ensure that each server farm is homogeneous (all real servers within a server farm have identical configurations)

Now looking at the issue you are facing, there is no way for the ACE to force the servers to send the 400 or to send a 400 response on its own. Since the server is sending the 400 I would recommend you to check if it accepts the way ACE reuses the connection.

If you have a packet capture we can analyze at packet level and check if anything changed between ACE and server.

Let me know if you have any questions.

Regards,
Siva

akhtar.samo Wed, 08/22/2012 - 00:57

Hi Siva,

We had a requirement from one of our customer that their client terminals (POS terminals) should be authenticated by the ACE which is terminating the SSL connection. Backend connections to the server is clear text.

In a normal SSL flow the server sends the certificate to the client and the client verifies the identity of the server but in our case we need ACE to authenticate the client or some form of mutual authentication should be there.

http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/vA2_3_0/configuration/ssl/guide/terminat.html#wp1117637

As per the documentation we have enabled the authgroup to enable the client authentication feature, but when we are testing the application it seems that only the front end (client to ACE) connection gets established but not the back end.

We have verified that if client authentication is disabled the application works fine but the ACE sends it the certificate and the client is not authenticated.

crypto authgroup POS

cert cert_client.pem

ssl-proxy service ssl-proxy

   key POS

   cert cert_server.pem

   authgroup POS

   ssl advanced-options POS

Our scenario is like given below with client authentication

(Server)---------------clear text----------------(ACE)-----------------SSL--------------------(POS Terminals/client)

Regards,

Akhtar

akhtar.samo Wed, 08/22/2012 - 01:43

Hello Siva,

Just wondering if this would be the right way to check traffic based on source IP before it gets loadbalanced ? These configuration doesn't work. Can you pls. assist.

**************************CONFIGURATION-OPTION-1*********************************************

rserver host PLATTS_APP

  ip address 192.168.0.1

  inservice

serverfarm host SF_PLATTS

  transparent

  rserver PLATTS_APP

    inservice

class-map  match-any SRC-IP-A

  2 match source-address 192.168.80.89 255.255.255.255

policy-map type loadbalance first-match PM_L7_BYPASS_SRC_IP

  class class-default

    serverfarm SF_PLATTS

class-map match-any CM_BYPASS_VIP

  2 match virtual-address 10.10.10.10 any

  3 match virtual-address 20.20.20.20 any

  4 match virtual-address 30.30.30.30 any

  5 match virtual-address 40.40.40.40 any

policy-map multi-match PM_BYPASS_SRC_IP

  class CM_BYPASS_VIP insert-before SRC-IP-A

    loadbalance vip inservice

    loadbalance policy PM_L7_BYPASS_SRC_IP

**************************CONFIGURATION-OPTION-2*********************************************

rserver host PLATTS_APP

  ip address 192.168.0.1

  inservice

serverfarm host SF_PLATTS

  transparent

  rserver PLATTS_APP

    inservice

class-map  match-any SRC-IP-A

  2 match source-address 192.168.80.89 255.255.255.255

policy-map type loadbalance first-match PM_L7_BYPASS_SRC_IP

  class SRC-IP-A

    serverfarm SF_PLATTS

class-map match-any CM_VIP

  2 match virtual-address 10.10.10.10 any

  3 match virtual-address 20.20.20.20 any

  4 match virtual-address 30.30.30.30 any

  5 match virtual-address 40.40.40.40 any

policy-map multi-match PM_

  class CM_BYPASS_VIP

    loadbalance vip inservice

    loadbalance policy PM_L7_

******************************************************************************

Regards,

Akhtar

sivaksiv Wed, 08/22/2012 - 03:03

Hi Akhtar,

Thanks for your question.

If I understand correctly, you want to setup SSL client authencation on ACE to setup mutual authentication between ACE and client. If you implement client authentication on ACE, it will send a client certificate request with all of the certificates in the authgroup. The cert that client sends in response must be signed by one of the certs in the cert group.

Since you mentioned that the front end connection is established but not the backend I would like to understand if the request atleast forwarded to the server in the backend after ssl authentication.

If you have a packet capture we can verify the TCP and SSL handshake between and client and ACE and handshake between ACE and server.

To configure load balancing based on SRC ip address,  the below config should work:

The traffic that matches 192.168.80.89 should be loadbalanced to SF_PLATTS

rserver host PLATTS_APP

  ip address 192.168.0.1

  inservice

serverfarm host SF_PLATTS

  transparent

  rserver PLATTS_APP

    inservice

class-map  match-any SRC-IP-A

  2 match source-address 192.168.80.89 255.255.255.255

policy-map type loadbalance first-match PM_L7_BYPASS_SRC_IP

  class class-default

    serverfarm SF_PLATTS

class-map match-any CM_BYPASS_VIP

  2 match virtual-address 10.10.10.10 any

  3 match virtual-address 20.20.20.20 any

  4 match virtual-address 30.30.30.30 any

  5 match virtual-address 40.40.40.40 any

policy-map multi-match PM_BYPASS_SRC_IP

  class SRC-IP-A

    loadbalance vip inservice

    loadbalance policy PM_L7_BYPASS_SRC_IP

  class CM_BYPASS_VIP

    loadbalance vip inservice

    loadbalance policy PM_L7_

Let me know if you have any questions.

Regards,
Siva

akhtar.samo Wed, 08/22/2012 - 04:04

Here is the challenge which we are facing,

1. If the traffic comes from SRC-IP-A and their destination is CM_BYPASS_VIP it should be loadbalanced using PM_L7_BYPASS_SRC_IP (using policy PM_BYPASS_SRC_IP)

2. If the traffic comes from SRC-IP-A and their destination is 'any' then it should using policy PM_MAIN_BCPROXY and in turn would be load balanced using PM_LB_SF_BCPROXY / serverfarm SF_BCPR

rserver host RS_BCPR01

  ip address 192.168.0.103

  inservice

rserver host RS_BCPR02

  ip address 192.168.0.104

  inservice

serverfarm host SF_PLATTS

  transparent

  rserver PLATTS_APP

    inservice

serverfarm host SF_BCPR

  transparent

  probe PROBE_TCP

  rserver RS_BCPR01

    inservice

  rserver RS_BCPR02

    inservice

sticky ip-netmask 255.255.255.255 address source STICKY-SOURCE

  replicate sticky

  serverfarm SF_BCPR

class-map match-all CM_SF_BCPR

  255 match virtual-address 0.0.0.0 0.0.0.0 tcp eq www

class-map  match-any SRC-IP-A

  2 match source-address 192.168.80.89 255.255.255.255

class-map match-any CM_BYPASS_VIP

  2 match virtual-address 10.10.10.10 any

  3 match virtual-address 20.20.20.20 any

  4 match virtual-address 30.30.30.30 any

  5 match virtual-address 40.40.40.40 any

policy-map type loadbalance http first-match PM_LB_SF_BCPROXY

  class class-default

    sticky-serverfarm STICKY-SOURCE

policy-map type loadbalance first-match PM_L7_BYPASS_SRC_IP

  class class-default

    serverfarm SF_PLATTS

policy-map multi-match PM_MAIN_BCPROXY

class CM_SF_BCPR

    loadbalance vip inservice

    loadbalance policy PM_LB_SF_BCPROXY

    loadbalance vip icmp-reply active

    appl-parameter http advanced-options PARAMAP_CASE

policy-map multi-match PM_BYPASS_SRC_IP

  class CM_BYPASS_VIP insert-before SRC-IP-A

    loadbalance vip inservice

    loadbalance policy PM_L7_BYPASS_SRC_IP

int vlan 300

service-policy input PM_BYPASS_SRC_IP

service-policy input PM_MAIN_BCPROXY

Regards,

Akhtar

sivaksiv Wed, 08/22/2012 - 13:30

Hi Akhtar,

I built a config that meets your requirement and tested working in my LAB.

Traffic that comes to CM_BYPASS_VIP with SRC ip 192.168.80.89 would be load balanced to PM_LB_SF_BCPROXY and traffic coming from SRC IP 192.168.80.89 to any would be loadbalanced to PM_L7_BYPASS_SRC_IP

==================== configuration ==============

rserver host RS_BCPR01

  ip address 192.168.0.103

  inservice

rserver host RS_BCPR02

  ip address 192.168.0.104

  inservice

serverfarm host SF_PLATTS

  transparent

  rserver PLATTS_APP

    inservice

serverfarm host SF_BCPR

  transparent

  probe PROBE_TCP

  rserver RS_BCPR01

    inservice

  rserver RS_BCPR02

    inservice

sticky ip-netmask 255.255.255.255 address source STICKY-SOURCE

  replicate sticky

  serverfarm SF_BCPR

class-map match-any CM_BYPASS_VIP

  2 match virtual-address 10.10.10.10 any

  3 match virtual-address 20.20.20.20 any

  4 match virtual-address 30.30.30.30 any

  5 match virtual-address 40.40.40.40 any

class-map match-all CM_SF_BCPR

  255 match virtual-address 0.0.0.0 0.0.0.0 tcp eq www

policy-map type loadbalance first-match PM_LB_SF_BCPROXY

  match test source-address 192.168.80.89 255.255.255.255

    sticky-serverfarm STICKY-SOURCE

policy-map type loadbalance first-match PM_L7_BYPASS_SRC_IP

  match test source-address 192.168.80.89 255.255.255.255

    sticky-serverfarm SF_PLATTS

policy-map multi-match NEW-POLICY

  class CM_BYPASS_VIP

    loadbalance vip inservice

    loadbalance policy PM_LB_SF_BCPROXY

    loadbalance vip icmp-reply active

    appl-parameter http advanced-options PARAMAP_CASE

  class CM_SF_BCPR

    loadbalance vip inservice

    loadbalance policy PM_L7_BYPASS_SRC_IP

    loadbalance vip icmp-reply active

    appl-parameter http advanced-options PARAMAP_CASE

int vlan 300

service-policy input NEW-POLICY

Let me know if you have any questions.

Regards,
Siva

akhtar.samo Thu, 08/23/2012 - 04:00

Thanks for your reply. Regarding client authentication, I want to know if this is the complete configuration required ?

We will be carrying out the tests one again to see if the front end or the back end connection is having problem.

May i know what would be the best way to find if the front end client authentication is successfull or not ?

Since this Expert Session is ending tomorrow, would it possible for you to send me your email id through PM so that I can contact you in case this client authentication issue is not getting fixed.

Regards,

Akhtar

sivaksiv Thu, 08/23/2012 - 06:56

Hi Akhtar,

No the configuration I have given is without client authentication and SSL. Add the below config to enable client authentication.

crypto authgroup POS

cert cert_client.pem

ssl-proxy service ssl-proxy

   key POS

   cert cert_server.pem

   authgroup POS

   ssl advanced-options POS

I sent you my contact information via PM.

Regards,

Siva

ferjbello Wed, 08/22/2012 - 12:37

Thanks siva, I think that the answer was good enough. No need to check the captures. Thanks a lot!!!!!

ferjbello Wed, 08/22/2012 - 12:44

Hi Siva,

Could you please give an scenario/example when to use SNAT? And another scenario/example using DNAT on the ACE?

thanks!!

Fernado

sivaksiv Thu, 08/23/2012 - 00:17

Hi Fernando,

Good question!

Source NAT: I can think of 2 different scenarios where it is primarily required:

1. One-arm mode:

One Arm Mode is used when not every server needs to be load balanced or if  you couldn’t move the gateway of all of the servers to the ACE when not all of them will be load balanced. One Arm Mode works very simply: the user hits the virtual IP address of the serverfarm on the ACE and then the ACE directs the traffic to the appropriate real server dependent upon which load balancing algorithm has been selected. The trick is getting return traffic from the real server to go back through the ACE; this is achieved with source NAT. With SRC NAT,  the user hits the ACE, src address is translated to one in the pool. The real server sees the source address as one in the pool and knows that that subnet resides on the ACEand replies back to the ACE. The ACE then NATs the address to the user’s real address and forwards the response.

Now on to setting up the source NAT pool, we have a good article on setting this up (complete with configs).

http://docwiki.cisco.com/wiki/Basic_Load_Balancing_Using_One_Arm_Mode_with_Source_NAT_on_the_Cisco_Application_Control_Engine_Configuration_Example

2. Routed-mode:

When you deploy ACE in Routed Mode where the real server gateway resides on an interface of the ACE. You would run into a issue where real servers hit the VIP for another serverfarm on the ACE and had the connection reset. The problem is that the load-balanced real-server talk directly to the originating server since they were in the same VLAN/subnet. So we need the load-balanced real-server to respond to the ACE and then from the ACE to the requesting server. For this to happen, you would setup Source NAT, when the connection from a real-server enters the ACE, it gets NAT’d to an address in the VIP subnet; when the real-server responds it responds to the ACE and then back to the requesting real-server.

Sample config:

Servers connecting to the VIP on VLAN YYY are load balanced AND source NAT'd to the VIP subnet.

class-map match-all REAL_SERVERS

2 match source-address x.x.x.0 255.255.255.0

class-map match-all VIP-X

2 match virtual-address y.y.y.5 tcp eq www

policy-map multi-match CLIENT_VIPS

  class VIP-X

    loadbalance vip inservice

    loadbalance policy SLB_LOGIC

    loadbalance vip icmp-reply active

  class REAL_SERVERS

    nat dynamic 1 vlan YYY

interface vlan XXX

  description Client vlan

  ip address x.x.x.1 255.255.255.0

  service-policy input CLIENT_VIPS

  no shutdown

interface vlan YYY

  description Servers vlan

  ip address y.y.y.1 255.255.255.0

  service-policy input CLIENT_VIPS

  nat-pool 1 y.y.y.2 y.y.y.2 netmask 255.255.255.0 pat

  no shutdown

------------------------------------------------------------------------------------------------

Destination NAT is ON by default on the ACE, as the VIP address will  be "translated" to the real server address.

If you want to just do a dst  NAT for a given ip in a scenario like below:

(Source,Destination)

Before NAT(x.x.x.x,y.y.y.y)====>After NAT(x.x.x.x,z.z.z.z)

Configure 1 rserver per serverfarm, where the real server will have your destination IP.

y.y.y.y will be your VIP address

z.z.z.z will be your rserver address

-------------------------------------------------------------------------------------------------

Let me know if you have any questions.

Regards,

Siva

ferjbello Thu, 08/23/2012 - 06:07

Just what I was looking for!

THanks so much!!!!

Actions

Login or Register to take actions

This Discussion

Posted August 3, 2012 at 12:27 PM
Stats:

Related Content

Discussions Leaderboard

Rank Username Points
1 1,551
2 369
3 333
4 228
5 212
Rank Username Points
5