4500 unstable CPU utilization 25% then 75%

Unanswered Question
Apr 11th, 2010

Dear all

i have 4507R with two Supervisor (IV WS-X4515)

i have strage CPU utilization it move from around 25%  to  around 70%  then again to around 25%

from the show processes cpu command

                           the change happen in this two processes only (this my only 2 unstable cpu process)

  Cat4k Mgmt HiPri
  Cat4k Mgmt LoPri

then i check with the command

show platform health

i notice this two thing which is always 30% and 10% , also no other things  higher then 4%

K2CpuMan Review       30.00  11.87     30     83  100  500   14  11   10  583:25
K2AccelPacketMan: Tx  10.00   1.86     20      0  100  500    2   1    1  81:28

S2w-JobEventSchedule  10.00   0.18     10      6  100  500    0   0    0  11:51
Stub-JobEventSchedul  10.00   2.06     10     76  100  500    2   2    1  113:16

i already check this doucment

http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml

which not help that much because

1- i am using MSTP ( my network is 180 switchs) my 4507 is the BACKBONE

2- i already use portfast with BPDU filter ( my Spanning tree proceess is stable with less the 2% cpu proceess)

3- i dont have IPX

4-i dont have any access-list with LOG keywork in it

finally i use this command

sh platform cpu packet statistics

here is part of the command


Packets Received by Packet Queue

Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Esmp                          19095201        55        48        43         34
Control                        2203955         7         1         2          0
Host Learning                 11682231         0         0         0         12
L3 Fwd Low                     1421178         3         0         1          0
L2 Fwd Low                    74638357       202       525       262        136
L3 Rx Low                      6333218        67        63        52         43
RPF Failure                         21         0         0         0          0
ACL fwd(snooping)              5466762        19        14        13         11
ACL log, unreach                139918         0         0         0          0
ACL sw processing                  146         0         0         0          0

i can't interpert this part of the out put or clear it without reloading the switch

can any one help me with that ?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
bhisham84 Wed, 04/14/2010 - 06:51

Hi,

The ouput you have given shows the normal cpu utilization.Please paste the output when cpu utilization is 75% so that we can come on the result which component is actully causing the cpu high utilization.

We have 2 main components which generaly cause High CPU utilization.

K2CpuMan:- It show CPU bound traffic.

K2L2:- These processes are responsible for maintenance of the various L2 tables.

A switch might experience high CPU utilization due to the Cat4k Mgmt LoPri process and the K2CpuMan and K2L2 Address Table reviews (using the show platform health command. High CPU utilization does not impact the traffic switched in hardware.

The problem is seen when a large MAC address table exists and when the switch is frequently relearning MAC addresses on multiple VLANs.

Plz check the loggs as well if any mac address flood is thr in.

Thanks/bhisham

shailesh.h Wed, 04/14/2010 - 07:49

Observe few things

1. capture the trend of CPU utilization with any SNMP tool monitoring

2. Capture and share logs of the switches

3. Check is there any frequent topology change either due to L-2 /L-3 loop or similar problem

4. share output of sh ver to see for bugs with IOS if any

Once you share we can have view what could be reason

ayman20012001 Sun, 04/18/2010 - 02:47

dear all

first Thank you  for ur replay , i realy appreachet your effort

before i  start  let me attech my show version .

i figure out something which make my CPU more stable but not yet normal 

As u all know that i m using MST spanning tree and i have more then 180 switch in my network my solution which i figure it out

is enable portfast in all user connection in all the 180 switch which help my because

as enabling portfast will not generate BPDU Topology change ( will make the switch in the clear the mac-address-table)

after doing this my process cpu is better but still up and down but with less frequency

check me output (attached) of

sh processes cpu | ex  0.00%  0.00%  0.00%

i keep entering this command  during the 20 sec of high cpu to make u see the output

this output  happen every around 90 sec as you see i keep entering this command   then it back to normal state around 35% process for 90 sec and so on

also regarding the output of this command

sh platform cpu packet statistics

my output


Queue                  Total           5 sec avg 1 min avg 5 min avg 1 hour avg
---------------------- --------------- --------- --------- --------- ----------
Esmp                          19700188        54        50        42         32
Control                        2636521         7         4         1          0
Host Learning                  4742955         0         0         0          2
L3 Fwd Low                      918530         3         2         0          0
L2 Fwd Low                    30051692       112       114       131         91
L3 Rx Low                      3555892        21        13        12          4
RPF Failure                          2         0         0         0          0
ACL fwd(snooping)              2635551        12         7         6          0
ACL log, unreach                 83459         0         0         0          0
ACL sw processing                    4         0         0         0          0

which i still don't understand why my L2 Fwd Low and Esmp is high

although my Host leaning is now appear to be normal

can any one let me know what is ESMP is ?

thanks

 

Actions

This Discussion