99% CPU on 6500

Unanswered Question
Mar 30th, 2011

Dears ,

I'm receiving about 99%CPU on 6500 .Butwhen i go show cpu sorted ,i can't see any process taking more than 2% load .

My IOS is s72033-ipservicesk9_wan-mz.122-18.SXF13.bin

6500#sh processes cpu sorted | exclude 0.00
CPU utilization for five seconds: 99%/97%; one minute: 99%; five minutes: 99%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
114       11136      1772       6284  0.79%  0.13%  0.30%   1 Virtual Exec
   6    84386920   6320367      13351  0.71%  0.27%  0.23%   0 Check heaps
123    94861244 849807118        111  0.31%  0.38%  0.39%   0 IP Input
180    10647916  52152600        204  0.07%  0.03%  0.02%   0 CEF process
316    12053884  68553405        175  0.07%  0.21%  0.08%   0 OSPF Router 1
281    31925228 239981938        133  0.07%  0.04%  0.05%   0 Port manager per
213    61331112 213008378        287  0.07%  0.08%  0.09%   0 SNMP ENGINE
  37     5577644  36376566        153  0.07%  0.05%  0.02%   0 Per-Second Jobs

Regards

Haris P

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 3 (2 ratings)
burleyman Wed, 03/30/2011 - 05:36

Please redo the command like this

show proc cpu sorted | ex 0.00% 0.00% 0.00%

and post the output.

Mike

gephelps Wed, 03/30/2011 - 11:20

The output won't change. The CPU is high due to interrupts and not processes which is why we do not see any process as being responsible. The 5 second CPU usage is reported as two numbers. The first is total cpu usage and the second is the amount due to interrupts. CPU under interrupt is typically due to packet switching.

On this platform most of the traffic should be forwarded in hardware. The question here would be what is this traffic which is being sent to the supervisor.

chatwithharis Thu, 03/31/2011 - 02:54

How I can know which traffic is send to CPU and which traffic is send to hardware ??

Also  is there is any ways to find the load on hardware ?

gephelps Wed, 03/30/2011 - 11:41

Look at this post by Adam:

https://supportforums.cisco.com/docs/DOC-15608

It  talks about netdr which is how one would determine what traffic is  being sent to the CPU. Once the traffic is known then you may be able to  figure out the cause of the punt based either on the traffic itself or  the configuration.

A few causes of high cpu on this platform are mentioned in the following link:

http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a00804916e0.shtml

burleyman Wed, 03/30/2011 - 12:14

George,

Thanks for the info it will help me in the future.

But the command I showed will give you different info. I ran in to this issue while trouble shooting a problem when I used show proc cpu sorted | e 0.00

It did not show me everything. I just did the two commands right after each other and you can see it shows different out put.

show proc cpu sorted | e 0.00
CPU utilization for five seconds: 9%/0%; one minute: 10%; five minutes: 10%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  27  2720477484 605939357       4489  5.27%  5.19%  5.10%   0 Cat4k Mgmt HiPri
  28  20298868682213348346          0  2.95%  3.42%  3.67%   0 Cat4k Mgmt LoPri
  38   489385396 177327540       2759  0.87%  0.90%  0.91%   0 Spanning Tree   
  16     8394432  28762515        291  0.23%  0.06%  0.01%   0 ARP Input

       
show proc cpu sorted | e 0.00%  0.00%  0.00%
CPU utilization for five seconds: 9%/0%; one minute: 10%; five minutes: 10%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  27  2720477524 605939543       4489  5.27%  5.19%  5.10%   0 Cat4k Mgmt HiPri
  28  20298868962213348382          0  2.95%  3.42%  3.67%   0 Cat4k Mgmt LoPri
  38   489385400 177327544       2759  0.87%  0.90%  0.91%   0 Spanning Tree   
  16     8394432  28762516        291  0.23%  0.06%  0.01%   0 ARP Input       
   5    46932392   6941969       6760  0.00%  0.07%  0.06%   0 Check heaps     
  25    14503816  52423964        276  0.00%  0.01%  0.01%   0 Per-Second Jobs 
  26    15753052   1105909      14244  0.00%  0.01%  0.00%   0 Per-minute Jobs 
  44    15655364  56634656        276  0.00%  0.02%  0.00%   0 IP Input        
  45    18088528 200537165         90  0.00%  0.04%  0.02%   0 CDP Protocol    
  50    15147028  83032015        182  0.00%  0.01%  0.01%   0 CEF process     
  67         132       216        611  0.00%  0.09%  0.02%   2 Virtual Exec    
  68     5507612 927466357          5  0.00%  0.01%  0.00%   0 PM Callback     
  82    34627936  31239309       1108  0.00%  0.00%  0.03%   0 SNMP ENGINE    

The first one could omit some key info, so safer to do the other command.

Mike

simionov.adrian Wed, 03/30/2011 - 13:25

I fixed exactly the same issue as discribed here by creating an ACL on the VTY interface, it was missing, and every couple of hours there was high cpu utilization. there was brute force on the ssh port of the VTY interface. the only difference is that was a different device model.

maybe can help you.

chatwithharis Thu, 03/31/2011 - 02:41

The issue is sorted now !!!!!. I have a packeteer connected to the LAN interface of this 6500 and there was a license limitation for BW

(600mbps ) on that packeteer and the 6500 was sending more traffic and when i reduced the traffic through that interface the CPU came down.

Now now my CPU is 1 to 2 % only . 6500 is still processing the same traffic .

The strange thing I noticed was ,i was getting packet drop from 6500 to one of my external link !!!which should not affect  by this issue with my internal connectivity problem

Actions

Login or Register to take actions

This Discussion

Posted March 30, 2011 at 2:58 AM
Stats:
Replies:7 Avg. Rating:3
Views:1215 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard