Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION CONTROL ENGINE

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to learn how to configure and troubleshoot the Application Control Engine with Cisco expert Gilles Dufour. Gilles is a software engineer for the Level 4 to Level 7 switches in the Internet Systems Business Unit since January 2005. He is a CCIE # 3878 in routing, switching and security.

Remember to use the rating system to let Gilles know if you have received an adequate response.

Gilles might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through July 11, 2008. Visit this forum often to view responses to your questions and the questions of other community members.

53 REPLIES
New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi,

I would like to have a expert opinion about ACE module configured in bridge mode vs routed mode, advantages and disadvantages.

I got some problems in one-arm mode (more complex configs), after all I changed to routed mode.

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Bridge mode offers the possibility to insert the ACE module transparently between servers and their gateway.

No need for re-addressing or changing servers routing table.

However, it is not always easy to troubleshoot and you need to keep in mind ACE will not source-nat traffic that is bridged.

Router mode will require re-addressing.

But it is easier to troubleshoot and no restriction in terms of source-nating.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi,

Could you please shed some light on what Static parse errors mean in the sh stats http output and how to debug what causes such errors.

Thanks

Andrei

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

ACE is a loadbalancer with lots of firewall feature. HTTP inspection can be turned on and OFF depending on how strict you want the blade to be.

However, even if turned off, ACE will still make sure the http header is valid.

One cause of static parse error is the presence of illegal characters in the url (non-ascii character).

Unfortunately there is no way to get more info from ACE itself.

A sniffer trace would be required to analyse it and identify the error.

If you can't find the problem inside the sniffer trace, I would recommened to open a service request with the TAC.

Thanks for your question.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

can i install FWSM in routed mode and ACE in bridged mode, so the gateway for servers will be the FWSM.

i tried that , but i can't ping from ACE to FWSM

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Yes, you can use a design with ACE in bridge mode and FWSM in router mode.

Can you ping from the servers to the FWSM through the ACE ?

When you ping from ACE, do you see a response from the FWSM coming back to ACE if you capture a sniffer trace of the ACE tengig interface ?

Thanks,

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Giles,

we are abt to design and implement ACE/FWSM

2 chasis with a module of each.

will it be good idea to use ace/fwsm both in routed mode.

wht failover methods to use for both

want the webserves and database servers to be in 2 diff vlans behind ace.

how the traffic from web server to database and vice versa shd be configured on ace.

there are other traffic types for these web servers and database servers. Like replication and patches. how wd ace cope with that because these traffic shdnt be loadbalance.how to configure?

wht kind of security features shd be enabled on ace for web servers and database servers.

Thanks in advance

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi,

thanks for your question.

Should the traffic between web servers and database servers go through the FWSM ?

If yes, bridge mode might be a better solution for the ACE module.

If yes, and you really want to guarantee that ACE will not *leak* traffic from the database vlan to the webserver vlan, you could also use different contexts.

That would make the config a little bit more difficult.

For the config, anything is possible.

I assume 1 vserver for the webserver and another vserver to the database.

Nothing particular here.

For the rest of the traffic, it really depends on your design. In routed mode, ACE will simply route your traffic from one vlan to the other like any router as long as you permit this traffic inside the access-group access-list.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi Gilles,

is it possible to replace the SSL-certificate the ACE 4710 Device Manager is using ? Out of the box the Device Manager is using a self-signed certificate - but I would like to use a certificate from our internal PKI.

Best Regards

Carsten

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi Gilles

I am using a nat-pool with single ip address (PAT) and it is assigned to a serverfarm for source Natting.

What command can I use on ACE to figure out how many current connections are Natted and How many more connections can be Natted by this Nat-pool.

Thanks

A.

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

That's a good question.

I usually do a 'show np [1|2] me-stat "-socm" | i NAT'

NAT Pool Alloc [addr]: 0 0

NAT Pool Alloc [addr/port]: 0 0

NAT Pool Free [addr]: 0 0

NAT Pool Free [addr/port]: 0 0

If you do pool_alloc_addr_port - free_addr_port you have the currently allocated ports.

The 64000 ports are equally splitted between the 2 IXP. So each get 32k ports.

If you are running out of ports, you should see the following counter incrementing :

NAT Pool Alloc [fail]: 0 0

Another way could be "show xlate | i x.x.x.x | count".

Gilles.

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Carsten,

technically we could access the linux shell and navigate the directory structure to locate and replace the current cert/key.

You would have to do this after every reboot because there is actually no command to do it.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

OK - thanks a lot for your fast reply. Do you now of any plans to change this ? As you might can imagine it's never a good idea to get used to accepting SSL security warnings.

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

there is no plan currently to change this as far as I can tell.

I'll introduce the idea to the product manager but it might be good for you to ask your Cisco sale/account representative to do the same.

Gilles

Bronze

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

I currently have 2 MS IIS web servers with SSL that I want to move over to the C6K ACE module. Is there a way to export the SSL and import it into the ACE?

Thx

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

The SSL Configuration guide is here

http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A1/configuration/ssl/guide/sslgd.html

And from there you can find the documentation on importing keys and certificates.

http://www.cisco.com/en/US/docs/interfaces_modules/services_modules/ace/v3.00_A1/configuration/ssl/guide/certkeys.html#wp1029280

So, you should be able to export the key/cert from the IIS server into a pkcs12 file.

I believe you then have to split the file in a key and a cert with openssl before you can import everything into the ACE module.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Thanks Giles for your response

is using ACE in one arm mode, running it in a routed mode.

This is what i m planning

1)MSFC--ace in one arm mode---firewall in routed mode---web servers

create a DMZ in fwsm for database servers

do you see any prblms here. MSFC will run ospf to exchange routes over WAN, only reason i m using ace in one arm is because if i need to patch my web servers or database severs, and replcation of database servers, it shall nt use ACE. if you think ACE is fine to be used as bridge , and will nt have any issues for above. i can use ACE is bridge and FWSM in routed mode (dmz for database)

i can only use 2 context on fwsm and i m using the other context for other set of traffic which will nt involve ACE at all.

what i must do for failover of all these components at aggregation layer. do i need to extend vlans or use ISL trunk between MSFC will be a good option

can you suggest on load balancing, sticky and probe methods. we r using oracle 11i. i have gone thru the doc but will need your opinion-Thanks

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

The problem is one-arm topology is that you need to be very careful to guarantee that the servers respond to the client going through the ACE module - no asymetry allowed.

Most people will enable client nat.

This will indeed for the reponse to go back to ACE.

However, this will also prevent your servers to know the client ip address.

All traffic will appear to be coming from a single ip belonging to the ACE module.

Another solution is policy-based routing but not all devices support it and it might be tricky to configure or troubleshoot.

I personally do not recommend one-armed mode unless the amount of traffic generated by the servers and that do not need to be loadbalanced is huge. In this case it would make sense to bypass ACE.

For failover, you usually use another chassis with the same modules and have a portchannel trunk between the 2 chassis.

ACE also requires a dedicated link for the FT traffic.

If your backup site is far away and L2 connectivity is not possible, you could also use Route Health Injection.

For an initial design I always recommend to use the default commands. So, roudrobin should be fine for loadbalancing.

Afte a while running the default, we can check the stats and see - based on your traffic - if another loadbalancing technic should be used.

The best sticky method for HTTP traffic would be cookie insert.

Probes should really be configured depending on your environment and what you consider important to monitor, what you consider acceptable failover time, ...

There is no magic config here.

Gilles.

Bronze

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

I have the following topology and running A2(1.1).

ASA--160---C6k(ACE with Vlan165/177 webfarm)

\160 (app/db serverfarm)

Vlan 160 is my inside serverfarm (ie apps and database)

Vlan 165 is the VIP Vlan

Vlan 177 is the Web servers

160 can ping 177 but cannot rdp

177 cannot ping or rdp into 160

outside clients to 165/177 is fine and 177 can ping to outside and have web access.

How do I troubleshoot or get 177 to access 160 in terms of accessing apps or db servers?

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Make sure there is no asymetric routing.

Capture sniffer trace in the different vlans and follow the path of the traffic.

If there is asymetry, you can do 'no norm' on all ACE interfaces.

But it would only fix issues related to routing.

If you have loadbalancing issues due to asymetry, it would not help. You will still have to fix the asymetry.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi Gilles

I have few questions for you

Q1. What is the difference between "ssl probe" and "SSL_PROBE_SCRIPT" script provided with ACE.

Q2. Is SSL_PROBE_SCRIPT in probe: directory any different from the script file available on CCO for 2.1 code.

(I am currently running 1.63 code and planning to upgrade to 2.1.)

Q3. If I upgrade my ACE from 1.63 to 2.1 Will the SSL_PROBE_SCRIPT and other scripts in probe: directory be upgraded as well?

Q4. If I upload a custom Script. Can it be used by multiple contexts? Do I need to upload it once and call it in different Contexts or I need to upload it n times for n contexts using unique names.

I am confused with the following statement in ACE user guide

"The filename that you assign the script must be unique across the contexts. You will use this filename when you load the script into the ACE memory and configure the probe"

Q5. Is there a way to monitor the utilization of NAT-Pools. I am using PAT with a single IP address for source NAT and dont want to drop connections due to absence of NAT resources.

Thanks in advance

Anthony.

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

A1: The ssl probe is a true SSL connection with an HTTP request. The script is just the translation of the CSM script which only does a SSL handshake.

A2: I believe they are the same.

A3: If the script needed to be modified because of known issues, yes they would be upgraded with the image. But I don't know of any issues with the SSL script. So it should be the same.

A4: You will need to copy it to every context.

I personally use the same file name in every context.

A5: I have answered the same question in this topic. Please refer to the previous answer.

Thanks,

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Thanks for your earlier responses.

I have few more in line of your responses.

Q Is there a way to copy a probe script from "disk0: of Admin context", to "disk0: of any other context". All of my other context are in one arm mode with no access to any ftp/tfp servers.

Q You mentioned 'show np [1|2] me-stat "-socm" | i NAT' command for looking at NAT stats. All "sh np x me-stat" commands outputs two columns. What does second column represents? Delta?

Q. Is there any detailed documnetation available on CCO that explains these me-stat commands?

Thanks

Anthony

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Anthony,

there is no command to copy files from one context to the other.

You can use the MSFC as a TFTP server.

Copy the file to the MSFC disk and then copy it from there to all the contexts.

The 2nd column is supposed to be the delta but it is acting weirdly right now. I would simply ignore it for now.

Those counters are *not yet* documented on CCO. But the nat counters should be very explicit.

In case you really need to know the exact meaning of a counter, post a question to the forum and I will certainly reply.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

As per my understanding "persistence rebalance" enables ACE to look at

every GET request in a single TCP connection and select appropriate server farm

as per the L7 criteria defined in LB policy maps.

How does "Persistence rebalance" works with Stickiness?

If I am using Source IP based stickiness and one client gets stuck on the basis of first

Get request, What will happen to the subsequent requests?

Is there any significance of using "Persistence rebalance" with source ip stickiness?

In the similar lines what happens if "Persistence rebalance" and "TCP Reuse" both are configured.

Thanks

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Persistence rebalance is needed when you have proxy servers connecting to your vip and you do stickyness based on payload information like cookie.

So, if you are using sticky srcip, it makes no sense to enable this option.

With tcp-reusem, persistence rebalance still has the same effect. Every request is treated independently. So for each request we will look if there is an existing tcp connection to the server that we can reuse.

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

Hi, Gilles

We have several questions:

In "cookie insert" persistence:

- Is the path cookie configuration supported?

- How is the policy configured so we can match the inserted cookie and maint persistence?

With CSS, the unbalanced traffic affects CPU consumption.

In the ACE firewall load balancing topology and two contexts (internal//external),

part of the traffic does not get balanced in the internal context for several reasons (e.g., direct traffic to servers)

Is the ACE performance affected?

In CSS it is possible to do SNAT with different IP by service.

We need to do so when the servers are located in different internal/external networks.

Is something similar supported by ACE?

Thanks

JM

Cisco Employee

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

There is no option to set the path.

The default is "/".

Simply create the sticky group

sticky http-cookie NAME MySticky

cookie insert

serverfarm MyServers

Then inside your http policy do

policy type loadbalance first-match MyPolicy

class class-default

sticky-serverfarm MySticky

You don't need to worry about the value of the cookie. ACE will take care of it for you.

ACE is more complex than the CSS.

There is 2 CPU and 16 MicroEngines.

The CPU will not be affected by routed traffic.

But the microengines will still be involved to switch/route the traffic.

So, there will be an impact but very little.

This routed traffic will also consume your Bandwidth. If you have a license for 4Gbps, make sure this routed traffic does not consume everything.

The nat-pool is selected with the outgoing interface.

So, if traffic is sent to server1, on interface A, you could use nat-pool N1.

If server is sent to server2, on interface B, you can use nat-pool N2.

It used to be the only way to have different nating.

With 2.1.1 you can also specify the nat-pool associated with a serverfarm.

switch/Admin(config)# policy-map type loadbalance first-match SF_Linux1

switch/Admin(config-pmap-lb)# class class-default

switch/Admin(config-pmap-lb-c)# nat ?

dynamic Configure dynamic network address translation

switch/Admin(config-pmap-lb-c)# nat dynamic ?

<1-2147483647> Specify network address-pool for translation

switch/Admin(config-pmap-lb-c)# nat dynamic 1 ?

vlan VLAN interface

switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 ?

serverfarm Specify serverfarm

switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm ?

backup Choose backup serverfarm for this NAT

primary Choose primary serverfarm for this NAT

switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm primary ?

Carriage return.

switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm primary

Gilles.

New Member

Re: ASK THE EXPERT - CONFIGURING/TROUBLESHOOTING THE APPLICATION

hi

Can ACE appliance be used a HeadEnd SSLv3 for non HTTP traffic. The current SSL headend is on OpenSSL linux from stunnel.org. can ACE decrypt SSL with certificates (just like HTTP) then sent the unside(cleartext) TCP traffic internally?

so the question is can SSL be used for generic TCP traffic? any limitations to that?

thanks

h

426
Views
19
Helpful
53
Replies