High CPU due to " IP SNMP " & " SNMP ENGINE "

mmendis · ‎11-09-2014

Hi All,

I have a Cisco 2851 router which placed on one of our company node. Currently it's handling 80M of traffic. Recently ?I have monitor increased in CPU due to SNMP. Appreciate if someone can help me with this ??

Following are the Logs taken from the router.

Router Version:-

"flash:c2800nm-adventerprisek9-mz.151-4.M8.bin"

2851# show proc cpu sorted
CPU utilization for five seconds: 84%/64%; one minute: 86%; five minutes: 75%
PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process
378     5408556    11245357        480 5.83% 6.46% 3.89%   0 IP SNMP
380     4739792     5616341        843 5.43% 6.49% 3.94%   0 SNMP ENGINE
142     3982236     8455561        470 3.67% 3.93% 2.80%   0 IP Input
379      936308     5615860        166 1.11% 1.25% 0.77%   0 PDU DISPATCHER
366      747504      735117       1016 0.71% 0.48% 0.49%   0 MRIB Trans
116       11196    14099948          0 0.55% 0.57% 0.55%   0 Ethernet Msec Ti
371      170552      782130        218 0.39% 0.27% 0.26%   0 MFIB_mrib_read
203      791500      983787        804 0.31% 0.38% 0.39%   0 ADJ resolve proc
370       10484     5557602          1 0.31% 0.27% 0.26%   0 MFIB_mrib_write
364      232568     2761087         84 0.23% 0.26% 0.25%   0 PIM Process
   2       70272       22341       3145 0.07% 0.06% 0.07%   0 Load Meter
189        9672      290274         33 0.07% 0.08% 0.07%   0 TCP Timer
313         920      558364          1 0.07% 0.02% 0.00%   0 Atheros LED Ctro
262       20168      140738        143 0.07% 0.01% 0.00%   0 OSPF-10 Router
106         568      111651          5 0.07% 0.00% 0.00%   0 COLLECT STAT COU
177        9160      151130         60 0.07% 0.07% 0.07%   0 CEF: IPv4 proces
289        6344     3487573          1 0.07% 0.07% 0.07%   0 MMON MENG
282       59712      223334        267 0.07% 0.06% 0.07%   0 BGP Router
372        8436       13966        604 0.07% 0.01% 0.00%   0 MFIB_rate
363        7600     1132583          6 0.07% 0.06% 0.07%   0 IGMP Input

set-2851#show snmp stats oid

time-stamp               #of times requested             OID
09:16:28 LK Nov 10 2014 327                     ifDescr
09:16:27 LK Nov 10 2014 398                     ifName
09:16:26 LK Nov 10 2014 396                     ifPhysAddress
09:16:26 LK Nov 10 2014 2                       ifLastChange
09:16:26 LK Nov 10 2014 2                       ifOperStatus
09:16:25 LK Nov 10 2014 548                     ipAddrEntry.2
09:16:23 LK Nov 10 2014 528                     ifType
09:16:09 LK Nov 10 2014 1                       internet.6.3.12.1.1
09:16:09 LK Nov 10 2014 1                       internet.6.3.11.2.1.3
09:16:09 LK Nov 10 2014 1                       internet.6.3.11.2.1.2
09:16:09 LK Nov 10 2014 1                       internet.6.3.11.2.1.1
09:16:09 LK Nov 10 2014 1                       internet.6.3.10.2.1.4
09:16:09 LK Nov 10 2014 1                       internet.6.3.10.2.1.3
09:16:09 LK Nov 10 2014 1                       internet.6.3.10.2.1.2
09:16:09 LK Nov 10 2014 1                       internet.6.3.10.2.1.1
09:16:09 LK Nov 10 2014 2                       internet.6.3.1.1.6.1
09:16:09 LK Nov 10 2014 1                       cbQosCMPrePolicyByte
09:16:09 LK Nov 10 2014 2                       cbQosCMPrePolicyByteOverflow
09:16:09 LK Nov 10 2014 1                       cbQosCMPrePolicyByteOverflow
09:16:09 LK Nov 10 2014 1                       cbQosCMPrePolicyByteOverflow
09:16:09 LK Nov 10 2014 5                       cbQosCMPrePolicyPkt64
09:16:09 LK Nov 10 2014 1                       cbQosPolicyMapDesc
09:16:09 LK Nov 10 2014 3                       cbQosPolicyMapName
09:16:09 LK Nov 10 2014 12                      cbQosParentObjectsIndex
09:16:09 LK Nov 10 2014 10                      cbQosObjectsType
09:16:09 LK Nov 10 2014 1                       cbQosObjectsType
09:16:09 LK Nov 10 2014 1                       cbQosObjectsType
09:16:09 LK Nov 10 2014 13                      cbQosConfigIndex

mmendis · ‎11-11-2014

Any Help ???

Seb Rupik · ‎11-12-2014

Hi there,

looking at your process stats of the 84% util, 64% of that is due to CPU interupts. The SNMP process being high up the list is a red herring. You should investigate what traffic is being sent to the CPU for processing.

Following the steps in these documents should aid in troubleshooting:

http://www.cisco.com/c/en/us/support/docs/routers/10000-series-routers/15095-highcpu.html

http://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/41120-highcpu-interrupts.html

cheers,

Seb.

mmendis · ‎11-12-2014

Thanks Seb !!!

But when I disable snmp-server community XXXX RO CPU utilization get down to 35 -40%... I suspected that one of my servers which using snmp to collect data may causing this problem... Any Ideas ????

jossabor · ‎12-03-2014

Taking a look you can see that the OIDs being polled the most are:

09:16:25 LK Nov 10 2014 548 ipAddrEntry.2
09:16:23 LK Nov 10 2014 528 ifType

So you can either check the monitoring tool using that community and increase the polling intervals or set a view to those OIDs.

Those are huge OIDs so it is expected that being polled a lot cause high cpu, on this case it is weird as they only have 5%, that is not high cpu at all, you can check the show process cpu history, I think about a bug showing wrong information on the processes.