Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

Cat6500: High CPU due to IOS SLB with NAT Server - Resolve with Dispatched Mode

Introduction

This document disucsses on high CPU condition seen in Catalyst 6500 platforms due to IOS Server Load Balancing (SLB) feature.

Problem Description:

Catalyst 6500 reporting high CPU due to interrupts and "IP Input" process.

C6K-A#show process cpu sorted | exclude 0.00
CPU utilization for five seconds: 98%/57%; one minute: 99%; five minutes: 97%
PID Runtime(ms)  Invoked      uSecs   5Sec   1Min   5Min TTY Process
227   816331652 12632718       4266 32.10% 31.69% 31.07%   0 IP Input

Troubleshooting and Symptoms:

(1)

The IOS SLB is configured with "nat server""

ip slb serverfarm TEST-FARM1

   nat server

   nat client POOL1

<snip>

(2)

Switch do NOT have any sw-installed Netflow entries:

C6K-A#show mls netflow ip sw-installed 
Displaying Netflow entries in Active Supervisor EARL in module 5
No Entries

C6K-A#show mls netflow ip sw-installed module 1 <<=== Ingress DFC module
Displaying Netflow entries in Active Supervisor EARL in module 1 No Entries

(3)

SLB has active connections:

C6K-A#show ip slb connections 
vserver         prot client                real               state        nat 
-------------------------------------------------------------------------------
VSERVER-NAME    TCP  <client-ip>:<port>    <real-ip>:<port>   <TCP-State>  S,C
<snip>

Root-Cause and Resolution:

When the client sends the traffic to virtual IP address, the load-balancer (in this case, IOS SLB) will NAT the traffic, as the real/physical severs are NOT aware of the virtual IP address.

Cat6500 with "nat server" configuration, the switch is NOT capable of creating hardware shortcuts. As a result, the traffic will be process/software switched. This is done by punting the traffic to the CPU and it can be verified by: (Here, 10.50.50.2 is IP address of a virtual server)

C6K-A#show tcam int vlan <client-vlan> acl in ip | inc 10.50.50.2
    punt         udp any host 10.50.50.2
    redirect     tcp any host 10.50.50.2 fragments
    policy-route tcp any host 10.50.50.2 eq <port#>
    <snip>

To resolve this issue, it is recommended to configure IOS SLB in "Dispatched" mode.

http://www.cisco.com/en/US/products/hw/modules/ps2706/products_configuration_example09186a0080094066.shtml

http://www.cisco.com/en/US/products/hw/routers/ps341/products_tech_note09186a0080134735.shtml

Further Information:

To know more on IOS SLB feature, please refer the 12.2SX Feature Configuration Guide:

http://www.cisco.com/en/US/docs/ios/12_2sx/feature/guide/slbsxf7.html

Please be aware that starting from 12.2(33)SXJ IOS SLB is NOT supported:

http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/release/notes/features.html#wp4798296

For feedback and comments, please contact Yogesh at yramdoss@cisco.com

Version history
Revision #:
1 of 1
Last update:
‎04-30-2013 10:48 AM
Updated by:
 
Labels (1)
Everyone's tags (2)