CCS11050 High CPU.

Unanswered Question

Please can you help me identify what is causing high CPU on a CSS11050 device?

Details of device are -

chdccss01# sh ver

Version: ap0610306s (6.10 Build 306)

Flash (Locked): 4.01 Build 2

Flash (Operational): 6.10 Build 306

Type: PRIMARY

Licensed Cmd Set(s): Standard Feature Set

Secure Management


chdccss01# sh chassis

Configuration for CSS 11150:


Name: CSS 11150 SW Version: 6.10 Build 306

HW Major Version: 03 HW Minor Version: 0

Serial Number: 21480063243

Base Mac Address: 00-10-58-03-50-45



Module Number Module Name Status


1 FEM primary

2 FEM primary

5 SCFM-PLUS primary


Port Number Port Name Status


1 e1 online

2 e2 online

3 e3 online

4 e4 online

5 e5 online

6 e6 online

7 e7 online

8 e8 online

9 e9 online

10 e10 online

11 e11 online

12 e12 online

13 e13 online

14 e14 online

15 e15 online

16 e16 online


chdccss01# sh system-resources

System Resources:

Installed Memory: 134,217,728 (128 MB)

Free Memory: 20,833,744 (19 MB)

CPU: 98%


Buffer Statistics:

Buffer Pool: 0

Size: 256 Total:1000 Available:1000 Failures: 0 Low Buffer Count: 688


Buffer Pool: 1

Size:1600 Total:9000 Available:8128 Failures: 0 Low Buffer Count: 7585



chdccss01#


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Gilles Dufour Fri, 12/07/2007 - 06:02

do the following.

go to debug mode - command is 'llama'

load symbol-table.

execute cpu hog.

Press 'd' several times to get a list of active processes.

then do unload symbol-table.


gduf-css11000-1(debug)# symbol-table load

Loading string table... 583,460

Loading symbols........ 21,561

Load complete.......... 1,100,924


gduf-css11000-1(debug)# cpu hog


CPU Hog Debug Options


d) Display data

Please enter selection [q=Quit]:d

Checking CPU Hog

TID Name Milliseconds

--- ---- ------------

0x836d92e0 PingKal 0

0x8768b4b0 tASSync 0

0x8375e5f0 AS_00000080.I 0

0x8584d3a0 consoleOB 0

0x876b9bc0 apRdyActvLed 0


Gilles Dufour Fri, 12/07/2007 - 07:12

all these processes are responsible to handle traffic.

Seems like you have reach some limit.


Get a 'flow stat' from llama mode so we can see amount of connections.


Gilles.

Thanks Giles,


Details are -


chdccss01(debug)# flow statistics


Flow Manager Statistics:

Cur High Avg

UDP Flows per second 22 502 15

TCP Flows per second 42 310 39

Total Flows per second 64 624 55

Hits per second 0 0 0

Number of Allocated Flows (non-purged) 3451

Number of Free Flows 94853

Number of Flow Drops 0

Max Number of Flow Control Blocks 159359

Number of Flows with SynAckPending 83


Accumulated Port Flow Statistics:

Current Number of Active Flows 3474

Total Flow Accounting Reports received 25690664

Total Out of Sequence Packet Received 165

Total Spoof Queue Mis-Hits 0



-------------------------------------------------------------------------------

Port CE Active Total Acct TCP UDP Rst FCBs

-------------------------------------------------------------------------------

#e1 1f00 2 7 5 2 0 0 15231

#e13 3001f00 1290 12715311 12714022 1027 263 0 26216

#e14 3401f00 0 12125 12121 0 0 0 26195

#e15 3801f00 1296 6241815 6240727 1171 125 0 26211

#e16 3c01f00 886 6722438 6723789 757 137 0 26214


chdccss01(debug)#


Cheers,

Colin

Gilles Dufour Fri, 12/07/2007 - 15:11

Colin,


that's not that much of flows, but it depends what kind of rules you have.

Any L7 rule ?

Do you have some acl logging ?

Do you have a lot of keepalives ?


I can confirm the processes consuming the cpu are the ones handling the traffic.

So, this box is running at its max.


Gilles.

Gilles,

Thankyou for the information. As a result we looked at the traffic flow in more detail through the CSS and it appears that we have some type of looping FIN/ACK traffic which is generating high volumes of small packets.

When this traffic is blocked the CSS CPU returns to normal (4-5%).

Thankyou for your assistance.

Kind Regards,

Colin.

Actions

This Discussion