Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to get on update on configuring and troubleshooting the ACE module and ACE 4710 with Cisco expert Sean Merrow.  Sean is an escalation engineer working in the Cisco Technical Assistance Center in Boxborough, Massachusetts. He has been with the Application Networking Services team for three and a half years. His areas of expertise include product configuration and support for the load balancing products, including the Cisco ACE module, ACE 4710 appliance, Global Site Selector, Content Services Switch, and Content Switching Module. He holds a bachelor's degree in information technology from the University of Massachusetts and CCIE certification (#25197) in Routing and Switching.

In addition, join Ask the Experts' first TweetChat with Sean Merrow on June 10, 2010 at 11 am PT.

Remember to use the rating system to let Sean know if you have received an adequate response.

Sean might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through June 18, 2010. Visit this forum often to view responses to your questions and the questions of other community members.

101 REPLIES
Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hello!

I have a couple of questions regarding DNSSEC. I’m close to beginning the first tests in order to shortly support it and I’d like to know what would I have to keep in mind when the time comes. I’m particularly concern about packet size (MTU), whether to use DNS inspect or not (and if it is DNSSEC-aware), fragments handling and recommendations. The specifics about DNSSEC, from the ACE point of view. If there is a document addressing this subject, please let me know.

Just so you know, I’ve run some tests to verify longer MTUs and they were successful but only using ICMP since I cannot simulate DNSSEC as you can imagine.

Thanks a lot!

Guido

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hello Guido,

DNSSEC load balancing is something that is being looked into on the ACE, but the ACE's current DNS inspection does not include DNSSEC.  Unless you need the ACE to look into the packets for something special, then I would think you could simply load balance it at layer 4 for now.  Do not use inspect dns on the DNSSEC traffic.  According to the ISC's page on DNSSEC Readiness, DNSSEC will also likely cause more fallbacks to TCP, so you would probably need to set up VIPs for both UDP and TCP, but this is quite common even for plain DNS.

I hope this helps,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Thanks, Sean!

I was thinking too on a plain L4 policy to keep it simple and efficient. We are expecting ~30k queries per second.

Considering the possibility of “larger packets or packets containing new DNSSEC record types” (DNSSEC Readiness), would you recommend some tuning to interface-level commands such as fragments or normalization? I believe jumbo MTU is a must, to be on the safe side. But, how about the other two?

I’m thinking just as far as ACE is concerned.

Thanks again!

Guido

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Guido,

Because DNSSEC is still fairly early in its adoption and rollout, along with the fact that the ACE doesn't directly support it yet, I';m not aware of any DNSSEC 'best practices' yet.  However, from a simple layer-4 TCP load balancing point of view, I can make the following recommendations:

  • Once you have some simulated DNSSEC traffic that you can test with, see how it works without making any fine tuning adjustments.  I like to keep things as simple as possible first, then only make changes as necessary.  I'm sure you agree.
  • If you run into problems, you can use the show fragment command to see if your problems require you to fine-tune the fragmentation configuration.
  • As for normalization, unless you are using DSR (direct server return) where the ACE will only see one direction of traffic, I would also leave it at its default.  Normalization mostly just makes sure that the protocol is not abusing the TCP protocol, which DNSSEC shouldn't do anyway.  You can always disable it if you think it might be affecting the traffic, then fine-tune from there. 

Regards,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Thank You for starting this discussion. It actaully comes at the perfect time as I am new the ACE platform.

My question is can you please explain the genreral configuration for load balancing if you wanted your vip to be on the same subnet as your real servers? Most of the configuration technotes and documents seem to have the vip and real servers on a different subnet.

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

Welcome to the ACE world!

Configuring the VIP to be in the same IP subnet as the real servers is not a problem at all.

MSFC/Router  ------------------------------ACE---------------------------Reals

                          vlan 10                             vlan 20

                       10.1.10.0/24                     10.1.20.0/24

In the above topology, you might have the following IP address allocations:

VIP:                        10.1.20.10

ACE VLAN 20 Int:    10.1.20.1

Real Server 1:          10.1.20.11

Real Server 2:          10.1.20.12

ACE VLAN 10 Int:    10.1.10.2

MSFC/Router Int:     10.1.10.1

MSFC/Router route:   ip route 10.1.20.0 255.255.255.0 10.1.10.2

As long as the MSFC/Router has a static route pointing to the ACE's VLAN 10 interface as the next hop to reach the VIP/Real IP subnet, then you should be good to go!

Hope this helps,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

This would be for either the ACE appliance or module..correct? I am working with the appliance model. Also you you have a doc you like for setting up basic load balancing, say testing out web services between a couple of real servers?

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

Yes, this would apply to both the module and appliance.  I good example config for basic load balancing can be found here.  The only change you would need to make is to use a VIP in the server subnet as discussed, and make sure the upstream router has a route to reach that subnet behind the ACE.

Regards,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Thanks for your reply. So if I follow the example in the link then for the network behind the ACE that network would only exist behind the ACE? Can you please give an example or good suggestion for the following type of setup?

6500 connected to the ACE with it's four switchports. The vlan/network for the real servers and vip is on the 6500 (along with other vlans). Can you please explain the setup when the vlan/network for the real servers and vip is actually on the 6500 connected to the ACE. I would assum the ACE being in routed mode you would use a netwotk that didn't exist on the 6500 for the vip.

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

I'm not quite clear one the topology that you are looking for.  But if the VIP and servers are on the same router (or Cat6500) upstream of the ACE 4710, then you might need to use a one-armed configuration.  With this config, the switch would have a layer-3 interface (SVI) on the same subnet as the VIP and reals, and it would forward traffic to the VIP.  The ACE would then load balance the traffic to the reals in the same subnet, but would also perform source-NAT so that the servers would see the connections sourced from a NAT address that the ACE owns.  This would force the servers to send their responses back to the ACE.  You can see an example of this here.

Does this help?  If not, please come back with more details about your requirements and topology.

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

I'll try and explain the topology/setup better (attached is a simple diagram). The subnet/vlan for the real servers (and what the vip would be since it would be from the same subnet/vlan as the real servers) lives on the 6500, and the ACE 4710 would hanging off the 6500. The actual physical servers are attached to the 6500. Of the implementations options available (assuming enough info is available) for basic load balancing would you recommend one over the others?

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

So it appears to me that you are using a one-armed configuration.  The four ports on the ACE 4710 are in a port-channel.  Client connections can come into the switch on any VLAN.  The switch will then route the client connections to the VIP.  The ACE will load balance the connections to the real servers.  The trick is to make sure that when the servers respond to a client connection, the response does not bypass the ACE and go directly to the client.  If all of this is correct, then the most recent configuration example that I gave to you should work fine.  It is a one-armed config with source-NAT.  You would just need to put the VIP on the server subnet.

Does this answer your question?

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

I think it does answer my question. Attached is a config, with the admin context, and "testcontext along with the 6500 switch.

One thing I am wondering is the following config portion in the one-armed example.

context VC_WEB
  allocate-interface vlan 100
  member RC_WEB                <-What is this referencing? Is it referencing something from the Admin context?

Thanks,
Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hey Brandon,

Your config looks pretty good.  You could collapse your management class-maps as shown below if you want:

class-map type management match-all MGT-ALLOW_CLASS
  2 match protocol https any
  4 match protocol icmp any
  6 match protocol telnet any
  8 match protocol ssh any

policy-map type management first-match REMOTE_MGMT_ALLOW_POLICY
  class MGT-ALLOW_CLASS
    permit

As for the following...

context VC_WEB
  allocate-interface vlan 100
  member RC_WEB

This would be in the Admin context.  It has to do with virtualization and how much of the physical resources of the ACE you want to allocate to the context.  So in the Admin context, you would first create a resource-class for each unique allocation of resources.  Then you would make each context a member of the resource-class that describes the amount of resources you want to apply to that context.  More than one context can be a member of a resource-class.  You can see an example that includes resource-class here.

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

The resource class...right. I now recognize the syntax. If I remember from the reading it was like either up to five (5) virtual context or resources classes using the base license?

One question for the Admin context management portion. In the example config I have the e-channel applied to all four (4) ACE interfaces. Could I (if I wanted) just set the first port (g1/1) on the ACE for a static vlan assignment (say vlan 10 for example) (and apply the e-channel to the other three (3) ACE ports for the server/client traffic allowing any/all vlans? Would the ACE let me do this if one port was set to a static vlan assignement for management (Admin context) while the remaining interfaces are assigned to the e-channel allowing ALL vlans? It seems there's documentation the ACE may give you an error message referring to the port (a vlan already assigned type of message) with the port for the static vlan assignment applied when you configure the e-channel allowing ALL vlans on the remaining or any other interface(s). I am thinkng it may work with the command "switchport trunk allowed vlan all" command. If you can do this do you see it as a viable setup?

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hey Brandon,

Yes, out-of-the-box, the ACE will support up to 5 contexts.  But this limit does not apply to resource-classes.

While you can certainly only include some of the 4 ports in your port-channel, as you pointed out, the ACE will not allow you to apply any one VLAN to more than one port.  For example, in my setup, I have one physical port used for FT and is connected to the peer ACE.  I have another physical port used only for management on VLAN 2 and is connected to an access port, as an access port, to the Cat.  I have the remaining two ports in a port-channel, also to the Cat.

For the ACE, it is not possible to apply ALL the VLANs to a trunk, as the most you can apply is 1024 VLANs:

ace-appliance-14/Admin(config-if)# switchport trunk allowed vlan ?
    Vlan string (10-100,230,400-420) (Max Size - 1024)

...and if I try to allocate my management VLAN 2 to my port-channel:

ace-appliance-14/Admin(config-if)# switchport trunk allowed vlan 2-1024
vlan 2 is already associated with GigabitEthernet 1/1

Hope this helps,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Thanks for this reply, it's very helpful.

From the looks of it would following be a possible setup?

ACE ports

g1/1 - connected to access port (on Cat) for management(we will say vlan 10)

g1/2 - connected to cat in ec

g1/3 - connected to cat in ec

g1/4 - not connected (reserved for future use)

Question is could I allow all vlans (2-9,11-1024) on the EC except vlan 10 since it's for mangement (as long as no server/client traffic for user context(s) (other than the Admin context) are from the management vlan 10). Does this look accurate?

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Brandon,

Yes, that sounds good to me.  Most ACE modules and appliances are deployed as a redundant pair, so leaving port 4 unused would be a wise decision so it can be used later for FT.  Current config would look something like this:

interface gigabitEthernet 1/1
  switchport access vlan 10
  no shutdown
interface gigabitEthernet 1/2
  channel-group 23
  no shutdown
interface gigabitEthernet 1/3
  channel-group 23
  no shutdown
interface gigabitEthernet 1/4
  shutdown
interface port-channel 23
  switchport trunk allowed vlan 2-9,11-1024
  no shutdown

HTH,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Very helpful, thanks.

One other question that comes to mind for curiosity sake. What if you have app servers (that are real servers of other context) on the same vlan/subnet as the ACE management Admin context (vlan 10 for our example)? Is there a work around solution, or is it simply best practice to have the management "Admin" context ip address of the ACE on a vlan/subnet other than any possible real servers or client vlans/subnets (if you are using an access vlan port on the ACE for the management interface), unless you are using the configuration where you have an EC applied to all ACE ports allowing ALL vlans with the management vlan being the native vlan the the EC. Hope this makes sense...;-).

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

Some customer's like to perform all their management from the Admin context, and hence, will only have a single VLAN interface in the Admin context used for management.  Only from the Admin context can you use the changeto command to move to other contexts.  Other customer's will use one or more of their client or server VLAN interfaces and simply apply their management service-policy to those.  It is fine to apply your management policy to the same interface that clients are coming on, or servers are off of.  Really it is a matter of preference, requirements, or just what makes most sense to you.

Does this answer your question, or have I missed anything?

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Yes, this does answer my question and thank you for giving the customer senarios for the ACE management.

Sean can you give a rough guestimate as the number of context for application load balancing can be setup per ACE (particularly with the base ACE license)? I know this will be a big "it depends". But say you want to put on your exchange environement and maybe a few applications each with several real servers for load balancing. Maybe a better way ask the question is...at what point would one need to consider aquiring a second ACE or upgrading thier base license? Just curious.

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

As you stated "it depends".   I have seen many customers use no more contexts than just the Admin context.  Theoretically, you could put all of your management and load balancing configuration just in the Admin context.  I have seen many very large configurations like this.  You could put many or all of your load balanced applications into a common load balancing context, and just use your Admin context for management (my personal preference).

There are a few ways to determine when it is time to obtain more ACE devices:

  • You can monitor the output of  show resource usage all from the Admin context.  If you see the Denies going up in any context, then you can try to modify your resource allocatoins.  If you have allocated them the best you can, then perhaps it's time to scale by adding more ACEs.  One recommendation is to allocate 20% of your resources to an unused context.  This will cause some denies when your ACE hits 80% of capacity.  At that point, you can reallocate some of this 20% to the context(s) that need them, but you can also start the planning to scale sideways with more ACE.
  • You can confirm that you are not approaching any of the limits of the ACE
  • If you find that you are reaching 80% capacity of the ACE, you'll want to scale so that you continue to have room for future growth.

I would say that most customer's are using 5 contexts or less, although many have had to upgrade their other licenses (SSL TPS, throughput, etc.)

Hope this helps,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Once more thank you for the reply, some very helpful info here.

A question regarding the use of vlan1 (particularly for management purposes). I have read and seen on forums that the use of vlan 1 on the ACE is not allowed (referring mostly to use as management). Does this sound accurate that you cannot use vlan1 (interface vlan1) for the Admin context for management? I understand the reasoning from a security perspective why you may not want to do this, but is correct? Could you please explain the reasoning behind this?

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon

You are correct.  The ACE prohibits the use of VLAN 1 by the administrator.  This applies to both the module and the 4710.  If you look at the output of the show service-policy command, you'll see VLAN 1 listed:

ace-appliance-14/Admin#  sho service-policy det

Policy-map : CLIENT_VIPS
Status     : ACTIVE
Description:  -----------------------------------------
Interface: vlan 1 2
   service-policy: CLIENT_VIPS
   :

   :

On the module, VLAN 1 is used for backplane communications between the Supervisor and the ACE.  On eht 4710, VLAN 1 is used between the main-board and the acceleration and optimization daughter-card.

Hope this clears it up.

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

A question regarding the (traffic forwarding/routing) setup when using the Admin context for management and having just one separate context for all load balanced applications in their own load balancing context (this setup does sound appealing).

Assuming a one-armed type of setup and multiple server farm's along with their real servers in your load balanced applications context with the real servers in each server farm setup on separate vlans/subnets connected to the 6500 (each server farm's vip is on the same vlan/subnet as their real servers), which approach would be best to ensure proper traffic forwarding/routing to/from each server farm/vip setup with-in the load balanced applications context to the 6500? It seems you may have static routes on the 6500 pointing to an svi with-in the load balanced applications context on the ACE for forwarding to the vip(s) with the static route(s) pointing toward an svi other than vip(s) vlan svi.

Picturing the following setup:

6500----------ACE,    real servers are attached to 6500, vip for real servers & server farms lives on ACE

ACE would have it's management (Admin context) allocated to an access vlan switchport g1/1 & g1/2-3 would be an EC to the 6500 for the load balancing applications context.

Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

I will assume that your real servers are using an SVI on the Cat6500 as their default gateway.  Correct me if I'm wrong there.  If the ACE is hanging off the Cat6500 in a one-armed configuration over the port-channel, then you would need to have the ACE do source NAT on the client IP addresses so you can force the real servers to send their responses back to the ACE, not bypass the ACE and go directly back to the clients.  You can find a good example of having the ACE off the the Cat in a one-armed config, and having the real servers also off of the Cat here.  It contains the configuration including the required source NAT.

Hope this helps,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

HI Sean,

The topology you picture is correct.

Regarding the source nat pool, couldn't you do a nat pool with just one address (where the example shows a pool of five addresses) for the real servers/ACE/client communication(s)?


Thanks,

Brandon

Silver

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Brandon,

Yes, as long as you include the pat keyword at the end of the nat-pool definition, you can use only address if you like.  This is quite common.  You can then add addresses to the pool later, if necessary.

HTH,

Sean

Community Member

Re: ASK THE EXPERTS - APPLICATION CONTROL ENGINE

Hi Sean,

Thanks, this setup (ACE one-armed) seems very similar to the standard SLB implementation that can be used on the higher end platforms like the 6500 (which I am familiar with), very similar. It the 6500 SLB setup I have used only one nat in the pool for all the client communications. Can you please explain when/why you would need to add more than one address in the nat pool? Nearly all configs with SLB on the 6500 I've seen only included one address in the pool.

Thanks,

Brandon

2529
Views
85
Helpful
101
Replies
CreatePlease to create content