12-07-2007 04:53 AM
Please can you help me identify what is causing high CPU on a CSS11050 device?
Details of device are -
chdccss01# sh ver
Version: ap0610306s (6.10 Build 306)
Flash (Locked): 4.01 Build 2
Flash (Operational): 6.10 Build 306
Type: PRIMARY
Licensed Cmd Set(s): Standard Feature Set
Secure Management
chdccss01# sh chassis
Configuration for CSS 11150:
Name: CSS 11150 SW Version: 6.10 Build 306
HW Major Version: 03 HW Minor Version: 0
Serial Number: 21480063243
Base Mac Address: 00-10-58-03-50-45
Module Number Module Name Status
1 FEM primary
2 FEM primary
5 SCFM-PLUS primary
Port Number Port Name Status
1 e1 online
2 e2 online
3 e3 online
4 e4 online
5 e5 online
6 e6 online
7 e7 online
8 e8 online
9 e9 online
10 e10 online
11 e11 online
12 e12 online
13 e13 online
14 e14 online
15 e15 online
16 e16 online
chdccss01# sh system-resources
System Resources:
Installed Memory: 134,217,728 (128 MB)
Free Memory: 20,833,744 (19 MB)
CPU: 98%
Buffer Statistics:
Buffer Pool: 0
Size: 256 Total:1000 Available:1000 Failures: 0 Low Buffer Count: 688
Buffer Pool: 1
Size:1600 Total:9000 Available:8128 Failures: 0 Low Buffer Count: 7585
chdccss01#
12-07-2007 06:02 AM
do the following.
go to debug mode - command is 'llama'
load symbol-table.
execute cpu hog.
Press 'd' several times to get a list of active processes.
then do unload symbol-table.
gduf-css11000-1(debug)# symbol-table load
Loading string table... 583,460
Loading symbols........ 21,561
Load complete.......... 1,100,924
gduf-css11000-1(debug)# cpu hog
CPU Hog Debug Options
d) Display data
Please enter selection [q=Quit]:d
Checking CPU Hog
TID Name Milliseconds
--- ---- ------------
0x836d92e0 PingKal 0
0x8768b4b0 tASSync 0
0x8375e5f0 AS_00000080.I 0
0x8584d3a0 consoleOB 0
0x876b9bc0 apRdyActvLed 0
12-07-2007 06:15 AM
12-07-2007 07:12 AM
all these processes are responsible to handle traffic.
Seems like you have reach some limit.
Get a 'flow stat' from llama mode so we can see amount of connections.
Gilles.
12-07-2007 07:57 AM
Thanks Giles,
Details are -
chdccss01(debug)# flow statistics
Flow Manager Statistics:
Cur High Avg
UDP Flows per second 22 502 15
TCP Flows per second 42 310 39
Total Flows per second 64 624 55
Hits per second 0 0 0
Number of Allocated Flows (non-purged) 3451
Number of Free Flows 94853
Number of Flow Drops 0
Max Number of Flow Control Blocks 159359
Number of Flows with SynAckPending 83
Accumulated Port Flow Statistics:
Current Number of Active Flows 3474
Total Flow Accounting Reports received 25690664
Total Out of Sequence Packet Received 165
Total Spoof Queue Mis-Hits 0
-------------------------------------------------------------------------------
Port CE Active Total Acct TCP UDP Rst FCBs
-------------------------------------------------------------------------------
#e1 1f00 2 7 5 2 0 0 15231
#e13 3001f00 1290 12715311 12714022 1027 263 0 26216
#e14 3401f00 0 12125 12121 0 0 0 26195
#e15 3801f00 1296 6241815 6240727 1171 125 0 26211
#e16 3c01f00 886 6722438 6723789 757 137 0 26214
chdccss01(debug)#
Cheers,
Colin
12-07-2007 03:11 PM
Colin,
that's not that much of flows, but it depends what kind of rules you have.
Any L7 rule ?
Do you have some acl logging ?
Do you have a lot of keepalives ?
I can confirm the processes consuming the cpu are the ones handling the traffic.
So, this box is running at its max.
Gilles.
12-13-2007 12:39 AM
Gilles,
Thankyou for the information. As a result we looked at the traffic flow in more detail through the CSS and it appears that we have some type of looping FIN/ACK traffic which is generating high volumes of small packets.
When this traffic is blocked the CSS CPU returns to normal (4-5%).
Thankyou for your assistance.
Kind Regards,
Colin.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: