process cpu is very high

santiago_figueroa · ‎01-08-2010

Hello I have a problem with process of CPU, it proccess is very high SNMP ENGINE

3825#sh process cpu his

3825 08:25:55 AM Friday Jan 8 2010 GDL

999999999999999999999999999999999999999999999999999999999999

100 ************************************************************

90 ************************************************************

80 ************************************************************

70 ************************************************************

60 ************************************************************

50 ************************************************************

40 ************************************************************

30 ************************************************************

20 ************************************************************

10 ************************************************************

0....5....1....1....2....2....3....3....4....4....5....5....6

0 5 0 5 0 5 0 5 0 5 0

CPU% per second (last 60 seconds)

999999999999999999999999999999999999999999999999999999999999

100 ############################################################

90 ############################################################

80 ############################################################

70 ############################################################

60 ############################################################

50 ############################################################

40 ############################################################

30 ############################################################

20 ############################################################

10 ############################################################

0....5....1....1....2....2....3....3....4....4....5....5....6

0 5 0 5 0 5 0 5 0 5 0

CPU% per minute (last 60 minutes)

* = maximum CPU% # = average CPU%

999921212221252274222212212212222122232221122221212211122221221212212212

999906295348410280154390080092315930240107960318081074714636499090882692

100 ###*

90 ###*

80 ###* *

70 ###* *

60 ###* *

50 #### * *

40 #### * **

30 #### * * ** * * * * * * * *

20 ####************************************************* ******************

10 #################################################***********############

0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..

0 5 0 5 0 5 0 5 0 5 0 5 0

CPU% per hour (last 72 hours)

* = maximum CPU% # = average CPU%

3825#sh processes cpu sorted

CPU utilization for five seconds: 99%/2%; one minute: 99%; five minutes: 99%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

166 14968004 99882 149859 90.76% 90.04% 91.33% 0 SNMP ENGINE

anyone can help me?

Thanks

Reza Sharifi · ‎01-08-2010

Hello Santiago,

Please refer to this document on how to troubleshot high CPU utilization caused by SNMP engine.

http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a00800948e6.shtml#high_cpu_ut

HTH

Reza

santiago_figueroa · ‎01-08-2010

Thank you for their quick answer . It is a great idea

marikakis · ‎01-08-2010

Hi Reza,

Useful document you posted (+5). Never came across it before. It also kind of explains why the routing table isn't printed sorted in CLI (as was discussed in another recent thread).

Thanks,

Maria

Reza Sharifi · ‎01-08-2010

Hi Maria,

Thanks for the rating!!!

Regards,

Reza

huangedmc · ‎01-08-2010

Santiago,

Could you please let us know if the CPU utilization has been stablized?

Was CEF turned on?

Based on your hostname, it's a 3800 ISR, whose CEF should be on by default.

I'm curious to know why SNMP caused CPU to spike, because we have a bunch of 3845's, just in case we run into the same problem one day, I'll know how to get it resolved.

thanks.

marikakis · ‎01-09-2010

Hi,

I didn't say a lot in my previous post, since the document posted by Reza pretty much included any ideas I had and more. If you are interested in resolution of such issues I can tell you I remember such an issue in 7500 series. It was a production router, and issue occurred at high traffic hours, so I didn't have much time for smart analysis and documentation. I just picked up the phone, called the NMS team, kindly asked them to stop whatever they came up doing that day, and guess what? Issue was magically resolved! They didn't tell me exactly what they were trying to do though.

However, as mentioned in the document posted by Reza, SNMP is a low priority process and theoretically more critical processes will run if needed. As mentioned in some thread in Network Management section by Joe Marcus Clarke, if available resources exist and aren't used for more critical things, SNMP might try to use them all to do its job. It is understandable though that nobody likes seeing the CPU of their devices hitting the roof. Joe suggests GETBULK requests to get a lot of information at once, instead of many small requests to the devices.

I would also be interested to know if this particular issue was resolved and what the cause was.

Kind Regards,

Maria