Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to learn how to configure and troubleshoot the Application Control Engine with Cisco expert Gilles Dufour. Gilles is a software engineer for the Level 4 to Level 7 switches in the Internet Systems Business Unit since January 2005. He is a CCIE # 3878 in routing, switching and security.
Remember to use the rating system to let Gilles know if you have received an adequate response.
Gilles might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through July 11, 2008. Visit this forum often to view responses to your questions and the questions of other community members.
I would like to have a expert opinion about ACE module configured in bridge mode vs routed mode, advantages and disadvantages.
I got some problems in one-arm mode (more complex configs), after all I changed to routed mode.
Bridge mode offers the possibility to insert the ACE module transparently between servers and their gateway.
No need for re-addressing or changing servers routing table.
However, it is not always easy to troubleshoot and you need to keep in mind ACE will not source-nat traffic that is bridged.
Router mode will require re-addressing.
But it is easier to troubleshoot and no restriction in terms of source-nating.
Could you please shed some light on what Static parse errors mean in the sh stats http output and how to debug what causes such errors.
ACE is a loadbalancer with lots of firewall feature. HTTP inspection can be turned on and OFF depending on how strict you want the blade to be.
However, even if turned off, ACE will still make sure the http header is valid.
One cause of static parse error is the presence of illegal characters in the url (non-ascii character).
Unfortunately there is no way to get more info from ACE itself.
A sniffer trace would be required to analyse it and identify the error.
If you can't find the problem inside the sniffer trace, I would recommened to open a service request with the TAC.
Thanks for your question.
can i install FWSM in routed mode and ACE in bridged mode, so the gateway for servers will be the FWSM.
i tried that , but i can't ping from ACE to FWSM
Yes, you can use a design with ACE in bridge mode and FWSM in router mode.
Can you ping from the servers to the FWSM through the ACE ?
When you ping from ACE, do you see a response from the FWSM coming back to ACE if you capture a sniffer trace of the ACE tengig interface ?
we are abt to design and implement ACE/FWSM
2 chasis with a module of each.
will it be good idea to use ace/fwsm both in routed mode.
wht failover methods to use for both
want the webserves and database servers to be in 2 diff vlans behind ace.
how the traffic from web server to database and vice versa shd be configured on ace.
there are other traffic types for these web servers and database servers. Like replication and patches. how wd ace cope with that because these traffic shdnt be loadbalance.how to configure?
wht kind of security features shd be enabled on ace for web servers and database servers.
Thanks in advance
thanks for your question.
Should the traffic between web servers and database servers go through the FWSM ?
If yes, bridge mode might be a better solution for the ACE module.
If yes, and you really want to guarantee that ACE will not *leak* traffic from the database vlan to the webserver vlan, you could also use different contexts.
That would make the config a little bit more difficult.
For the config, anything is possible.
I assume 1 vserver for the webserver and another vserver to the database.
Nothing particular here.
For the rest of the traffic, it really depends on your design. In routed mode, ACE will simply route your traffic from one vlan to the other like any router as long as you permit this traffic inside the access-group access-list.
is it possible to replace the SSL-certificate the ACE 4710 Device Manager is using ? Out of the box the Device Manager is using a self-signed certificate - but I would like to use a certificate from our internal PKI.
I am using a nat-pool with single ip address (PAT) and it is assigned to a serverfarm for source Natting.
What command can I use on ACE to figure out how many current connections are Natted and How many more connections can be Natted by this Nat-pool.
That's a good question.
I usually do a 'show np [1|2] me-stat "-socm" | i NAT'
NAT Pool Alloc [addr]: 0 0
NAT Pool Alloc [addr/port]: 0 0
NAT Pool Free [addr]: 0 0
NAT Pool Free [addr/port]: 0 0
If you do pool_alloc_addr_port - free_addr_port you have the currently allocated ports.
The 64000 ports are equally splitted between the 2 IXP. So each get 32k ports.
If you are running out of ports, you should see the following counter incrementing :
NAT Pool Alloc [fail]: 0 0
Another way could be "show xlate | i x.x.x.x | count".
technically we could access the linux shell and navigate the directory structure to locate and replace the current cert/key.
You would have to do this after every reboot because there is actually no command to do it.
OK - thanks a lot for your fast reply. Do you now of any plans to change this ? As you might can imagine it's never a good idea to get used to accepting SSL security warnings.
there is no plan currently to change this as far as I can tell.
I'll introduce the idea to the product manager but it might be good for you to ask your Cisco sale/account representative to do the same.
I currently have 2 MS IIS web servers with SSL that I want to move over to the C6K ACE module. Is there a way to export the SSL and import it into the ACE?
The SSL Configuration guide is here
And from there you can find the documentation on importing keys and certificates.
So, you should be able to export the key/cert from the IIS server into a pkcs12 file.
I believe you then have to split the file in a key and a cert with openssl before you can import everything into the ACE module.
Thanks Giles for your response
is using ACE in one arm mode, running it in a routed mode.
This is what i m planning
1)MSFC--ace in one arm mode---firewall in routed mode---web servers
create a DMZ in fwsm for database servers
do you see any prblms here. MSFC will run ospf to exchange routes over WAN, only reason i m using ace in one arm is because if i need to patch my web servers or database severs, and replcation of database servers, it shall nt use ACE. if you think ACE is fine to be used as bridge , and will nt have any issues for above. i can use ACE is bridge and FWSM in routed mode (dmz for database)
i can only use 2 context on fwsm and i m using the other context for other set of traffic which will nt involve ACE at all.
what i must do for failover of all these components at aggregation layer. do i need to extend vlans or use ISL trunk between MSFC will be a good option
can you suggest on load balancing, sticky and probe methods. we r using oracle 11i. i have gone thru the doc but will need your opinion-Thanks
The problem is one-arm topology is that you need to be very careful to guarantee that the servers respond to the client going through the ACE module - no asymetry allowed.
Most people will enable client nat.
This will indeed for the reponse to go back to ACE.
However, this will also prevent your servers to know the client ip address.
All traffic will appear to be coming from a single ip belonging to the ACE module.
Another solution is policy-based routing but not all devices support it and it might be tricky to configure or troubleshoot.
I personally do not recommend one-armed mode unless the amount of traffic generated by the servers and that do not need to be loadbalanced is huge. In this case it would make sense to bypass ACE.
For failover, you usually use another chassis with the same modules and have a portchannel trunk between the 2 chassis.
ACE also requires a dedicated link for the FT traffic.
If your backup site is far away and L2 connectivity is not possible, you could also use Route Health Injection.
For an initial design I always recommend to use the default commands. So, roudrobin should be fine for loadbalancing.
Afte a while running the default, we can check the stats and see - based on your traffic - if another loadbalancing technic should be used.
The best sticky method for HTTP traffic would be cookie insert.
Probes should really be configured depending on your environment and what you consider important to monitor, what you consider acceptable failover time, ...
There is no magic config here.
I have the following topology and running A2(1.1).
ASA--160---C6k(ACE with Vlan165/177 webfarm)
\160 (app/db serverfarm)
Vlan 160 is my inside serverfarm (ie apps and database)
Vlan 165 is the VIP Vlan
Vlan 177 is the Web servers
160 can ping 177 but cannot rdp
177 cannot ping or rdp into 160
outside clients to 165/177 is fine and 177 can ping to outside and have web access.
How do I troubleshoot or get 177 to access 160 in terms of accessing apps or db servers?
Make sure there is no asymetric routing.
Capture sniffer trace in the different vlans and follow the path of the traffic.
If there is asymetry, you can do 'no norm' on all ACE interfaces.
But it would only fix issues related to routing.
If you have loadbalancing issues due to asymetry, it would not help. You will still have to fix the asymetry.
I have few questions for you
Q1. What is the difference between "ssl probe" and "SSL_PROBE_SCRIPT" script provided with ACE.
Q2. Is SSL_PROBE_SCRIPT in probe: directory any different from the script file available on CCO for 2.1 code.
(I am currently running 1.63 code and planning to upgrade to 2.1.)
Q3. If I upgrade my ACE from 1.63 to 2.1 Will the SSL_PROBE_SCRIPT and other scripts in probe: directory be upgraded as well?
Q4. If I upload a custom Script. Can it be used by multiple contexts? Do I need to upload it once and call it in different Contexts or I need to upload it n times for n contexts using unique names.
I am confused with the following statement in ACE user guide
"The filename that you assign the script must be unique across the contexts. You will use this filename when you load the script into the ACE memory and configure the probe"
Q5. Is there a way to monitor the utilization of NAT-Pools. I am using PAT with a single IP address for source NAT and dont want to drop connections due to absence of NAT resources.
Thanks in advance
A1: The ssl probe is a true SSL connection with an HTTP request. The script is just the translation of the CSM script which only does a SSL handshake.
A2: I believe they are the same.
A3: If the script needed to be modified because of known issues, yes they would be upgraded with the image. But I don't know of any issues with the SSL script. So it should be the same.
A4: You will need to copy it to every context.
I personally use the same file name in every context.
A5: I have answered the same question in this topic. Please refer to the previous answer.
Thanks for your earlier responses.
I have few more in line of your responses.
Q Is there a way to copy a probe script from "disk0: of Admin context", to "disk0: of any other context". All of my other context are in one arm mode with no access to any ftp/tfp servers.
Q You mentioned 'show np [1|2] me-stat "-socm" | i NAT' command for looking at NAT stats. All "sh np x me-stat" commands outputs two columns. What does second column represents? Delta?
Q. Is there any detailed documnetation available on CCO that explains these me-stat commands?
there is no command to copy files from one context to the other.
You can use the MSFC as a TFTP server.
Copy the file to the MSFC disk and then copy it from there to all the contexts.
The 2nd column is supposed to be the delta but it is acting weirdly right now. I would simply ignore it for now.
Those counters are *not yet* documented on CCO. But the nat counters should be very explicit.
In case you really need to know the exact meaning of a counter, post a question to the forum and I will certainly reply.
As per my understanding "persistence rebalance" enables ACE to look at
every GET request in a single TCP connection and select appropriate server farm
as per the L7 criteria defined in LB policy maps.
How does "Persistence rebalance" works with Stickiness?
If I am using Source IP based stickiness and one client gets stuck on the basis of first
Get request, What will happen to the subsequent requests?
Is there any significance of using "Persistence rebalance" with source ip stickiness?
In the similar lines what happens if "Persistence rebalance" and "TCP Reuse" both are configured.
Persistence rebalance is needed when you have proxy servers connecting to your vip and you do stickyness based on payload information like cookie.
So, if you are using sticky srcip, it makes no sense to enable this option.
With tcp-reusem, persistence rebalance still has the same effect. Every request is treated independently. So for each request we will look if there is an existing tcp connection to the server that we can reuse.
We have several questions:
In "cookie insert" persistence:
- Is the path cookie configuration supported?
- How is the policy configured so we can match the inserted cookie and maint persistence?
With CSS, the unbalanced traffic affects CPU consumption.
In the ACE firewall load balancing topology and two contexts (internal//external),
part of the traffic does not get balanced in the internal context for several reasons (e.g., direct traffic to servers)
Is the ACE performance affected?
In CSS it is possible to do SNAT with different IP by service.
We need to do so when the servers are located in different internal/external networks.
Is something similar supported by ACE?
There is no option to set the path.
The default is "/".
Simply create the sticky group
sticky http-cookie NAME MySticky
Then inside your http policy do
policy type loadbalance first-match MyPolicy
You don't need to worry about the value of the cookie. ACE will take care of it for you.
ACE is more complex than the CSS.
There is 2 CPU and 16 MicroEngines.
The CPU will not be affected by routed traffic.
But the microengines will still be involved to switch/route the traffic.
So, there will be an impact but very little.
This routed traffic will also consume your Bandwidth. If you have a license for 4Gbps, make sure this routed traffic does not consume everything.
The nat-pool is selected with the outgoing interface.
So, if traffic is sent to server1, on interface A, you could use nat-pool N1.
If server is sent to server2, on interface B, you can use nat-pool N2.
It used to be the only way to have different nating.
With 2.1.1 you can also specify the nat-pool associated with a serverfarm.
switch/Admin(config)# policy-map type loadbalance first-match SF_Linux1
switch/Admin(config-pmap-lb)# class class-default
switch/Admin(config-pmap-lb-c)# nat ?
dynamic Configure dynamic network address translation
switch/Admin(config-pmap-lb-c)# nat dynamic ?
<1-2147483647> Specify network address-pool for translation
switch/Admin(config-pmap-lb-c)# nat dynamic 1 ?
vlan VLAN interface
switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 ?
serverfarm Specify serverfarm
switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm ?
backup Choose backup serverfarm for this NAT
primary Choose primary serverfarm for this NAT
switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm primary ?
switch/Admin(config-pmap-lb-c)# nat dynamic 1 vlan 20 serverfarm primary
Can ACE appliance be used a HeadEnd SSLv3 for non HTTP traffic. The current SSL headend is on OpenSSL linux from stunnel.org. can ACE decrypt SSL with certificates (just like HTTP) then sent the unside(cleartext) TCP traffic internally?
so the question is can SSL be used for generic TCP traffic? any limitations to that?