Virtual Context Resource Allocation Best Practices for Cisco ACE

[Sandeep Singh]

• Introduction
• Virtual Contexts
• Allocation Groups
• Overall Allocation
• Sticky Resources
• Syslog Buffer Resources
• Management Connection Resources
• Related Information

Introduction

Using ACE you can create a virtual environment, called a virtual context, in which a single ACE appears as multiple virtual devices. Each of these context is configured and managed independently. By default ACE provides an Admin context, with the ability to define up to five user contexts. Upto 20 contexts can be defined with additional licenses. Although virtualization allows you to create multiple contexts, actually you have a single ACE with finite resources, such as the number of concurrent connections. To address this limitation ACE provides resource classes that allows you to manage each virtual context's access to physical ACE resources. A resource class is a definition of what portion of an ACE's overall resources will be assigned, at a minimum or maximum, to any given context. One resource class may be associated with one or more contexts.

Virtual Contexts

With ACE virtualization, you can create a virtual environment, called a virtual context, in which a single ACE appears as multiple virtual devices, each configured and managed independently. A virtual context allows you to closely and efficiently manage system resources, ACE users, and the services that you provide to your customers. The ACE is preconfigured with a default resource class for the Admin context. This default resource class is applied to all virtual contexts that you create. It allows a maximum of 100 percent access to all resources by all virtual contexts. When a resource is being used to its maximum limit, the ACE will deny additional requests for that resource from any other virtual contexts. To avoid oversubscribing resources and to help guarantee that resource availability is shared among multiple virtual contexts, you create custom resource classes and associate them with the virtual contexts you define..

Allocation Groups

ACE resources are allocated within a resource-class and we can consider them divided into two main groups. The “quantity” group resources are where a finite amount is defined. When this amount is reached no new connection using this resource can be created. The “rate” group resource are for which we refer to resources per second.

The first group contains:

1) acl-memory-Limits: The memory space allocated for ACLs.

2) buffer-Limits: The number of syslog buffers.

3) conc-connections-Limits: The number of simultaneous connections.

4) mgmt-connections-Limits: The number of management (to-the-ACE) connections.

5) proxy-connections-Limits: The number of proxy connections.

6) regexp-Limits: The amount of regular expression memory. Double wildcard statements use much more memory than other single wildcard statements. A double wildcard regex is one where you have a wildcard (.*) before and after a string. It is recommended to have no more than 10 double wildcard regex statements per ACE. When the VIP class-map is created and then the regex class-map to be tied to the policy-map, the VIP requested the regexp memory for the regex class-map. So in order to free that memory, you have to remove the VIP class-maps.

7) sticky-Limits: The number of entries in the sticky table. You must configure a minimum value for sticky to allocate resources for sticky database entries.  You can allocate resources to sticky by either configuring a minimum percentage of resources specifically for sticky (limit-resource sticky) or by configuring a minimum percentage of resources for all (limit-resource all).

8) xlates-Limits: The number of network and port address translation entries.

The second group uses the keyword "rate" and it limits the resource as a number per second for the following:

1) connections-Limits: The number of connections per second of any kind.

2) inspect conn-Limits: The number of application protocol inspection connections per second for File Transfer Protocol (FTP) and Real-Time Streaming Protocol (RTSP) only.

3) mac-miss-Limits: The ACE traffic sent to the control plane when the encapsulation is not correct in bytes per second.

4) ssl-bandwidth-Limits: The number of SSL connections per second.

5) syslog-Limits: The number of syslog messages per second.

6) bandwidth-Limits: Total ACE throughput in bytes per second for one or more contexts. The maximum bandwidth rate per context is determined by your bandwidth license. By default the entry-level ACE has a 4Gbps through-traffic bandwidth and a 1Gbps management-traffic bandwidth for a total maximum bandwidth of 5 Gbps. You can upgrade the ACE with an optional 8Gbps or 16Gbps bandwidth license. When you configure a minimum bandwidth value for a resource class in the ACE, the ACE subtracts that configured value from the total bandwidth maximum value of all contexts in the ACE, regardless of the resource class with which they are associated. The total bandwidth rate of a context consists of the following two components:

7) throughput - Limits through-the-ACE traffic. This is a derived value (you cannot configure it directly) and it is equal to the bandwidth rate minus the mgmt-traffic rate for the 4Gbps and 8Gbps licenses. With a 16Gbps license, this value is calculated slightly differently.

8) mgmt-traffic - Limits management (to-the-ACE) traffic in bytes per second. This parameter is independent of the limit-resource all minimum command. To guarantee a minimum amount of management traffic bandwidth, you must explicitly allocate a minimum percentage to management traffic using the limit-resource rate mgmt-traffic minimum command.

There is a way to configure all resources at once using the “all resource”. This limits all resources to the specified value for all contexts assigned to this resource class, except for management traffic bandwidth. Management traffic bandwidth remains at the default values until you explicitly configure a minimum value for management traffic.

Note that the limit that you set for individual resources when you use the limit-resource command overrides the limit that you set for all resources when you use the limit-resource all command.

Overall Allocation

It is a good design if ACE installations do not run upto their full capacity, as the reserve capacity can later on be used for more scaling. To accomplish this goal create a reserved resource class with a guarantee of 20 to 40 percent of all the ACE resources and configure a virtual context dedicated solely to ensuring that these resources are reserved. Then you can efficiently distribute such reserved resources to contexts as capacity demands for handling client traffic increase over time. Other than this specific reserved context it is recommended not to create/retain any unused context with or without resource allocation.

To achieve this use the following commands:

resource-class capacity-planning
  limit-resource all minimum 20.00 maximum equal-to-min
  limit-resource rate mgmt-traffic minimum 20 maximum equal-to-min

This recommendation assumes that there is one context free to use as resource blocker. If there is no free context (because a low number of context license is used), you can use the Admin context to block resources, only if the Admin context is not used for traffic. To do this configure the same resource class than above but with maximum unlimited and link it to the Admin context. If you have traffic running through the Admin context then you should allocate its resources accordingly and not follow the above.

Sticky Resources

To be able to use stickiness inside one ACE context, the context must have the sticky resources allocated. The default resource allocation is not enough. In ACE code 1.x, the sticky resources is not part of the “all” resources. Therefore you need to specify the sticky allocation explicitly.

resource-class test
  limit-resource all minimum 1.00 maximum equal-to-min
  limit-resource sticky minimum 1.00 maximum equal-to-min

In ACE code 2.x the sticky resource is now part of the “all” resources and only one line is needed if you want to assign all resources the same way:

resource-class test
  limit-resource all minimum 1.00 maximum unlimited

The sticky resource always uses the “maximum equal-to-min” even if you configured “maximum unlimited”.

Syslog Buffer Resources

The size of the total syslog buffer on the ACE is fixed at 4,000,000 bytes. When a part of syslog buffer is assigned to a context, a part of this memory is used by the context. The log is circular in the allocated space; once the space is full the newer message will replace the oldest one. If the syslog buffer resource for one context is assigned with maximum unlimited, then this context is capable of filling the full available syslog space.

For example, if you have three contexts and you have assigned buffer as "maximum unlimited" then as long as there is free buffer space, the buffer for each context will keep increasing. Once there is not more free buffer then the circulation will begin. The buffer only increases until it reaches the max memory and starts circulating. When a resource is fully utilized it is not possible to create a new context that uses this resource. For the syslog buffer, it is also impossible to modify resource usage if one of the contexts has maxxed out its SYSLOG buffer resource as you cannot decrease a resource in use. You will then need to clear the syslog in one of the context to free enough resources to assign it to a 4th context. To avoid this it is recommended to always use "maximun equal-to-min" when allocating syslog buffer resources.

Management Connection Resources

The “mgmt-connections” parameter not only counts the number of management (to-the-ACE) connections. It also includes the probes connections as they are initiated from the ACE. An individual probe requires 2 management connections.  If however the "passdetect interval" is reduced to under 33 secs then multiple tcp connections will be consumed as probes will overlap. In this scenario the available management connections can be depleted and services will go offline due to probe failures if not enough mgmt-connections are allocated to the context.

There is a difference between ACE 1.x code and ACE 2.x code:
ACE 1.0 maximum of 5,000 (five thousand) management connections
ACE 2.0 - maximum of 100,000 (one hundred thousand) management connections

In ACE 1.0 code assigning 10% max of mgmt connections to one context means only 500 connections, so less than 250 simultaneous probes. Monitor the use of this resource by the ACE probes. The main information to keep track of is the number of denied connection that shows a resource allocation issue. If denied connections are increasing then you should consider allocating more resource to that context. In any case it is best to keep the mgmt-connections resource as maximum unlimited.

Related Information

Configuring Virtual Context on ACE

ACE Troubleshooting Guide - Managing Resources

[lboisvert2]

Hi,

What's the best practice rule for the "to-cp-ipcp rate" value? What is it use for? I didn't see it in your document (very usefull by the way).

Thank you,

Louis Boisvert