SNMP High CPU

Unanswered Question
Feb 6th, 2010

hi all, one of our router starting getting high cpu due to snmp.

i've enabled an acl to filter all inboud snmp on a heavily used public interface but the problem is sill there.

any other way to locate the culprit? there is no possible way that we can enable a sniffer as many interfaces are tied into the public side.

Router#sh snmp

69622828 SNMP packets input
    0 Bad SNMP version errors
    996 Unknown community name
    4648493 Illegal operation for community name supplied <-- increasing
    22 Encoding errors
    354120083 Number of requested variables <-- increasing
    0 Number of altered variables
    63480557 Get-request PDUs
    5951761 Get-next PDUs
    198 Set-request PDUs
    0 Input queue packet drops (Maximum queue size 1000)
69743297 SNMP packets output
    0 Too big errors (Maximum packet size 1500)
    57979 No such name errors
    0 Bad values errors
    0 General errors
    69621693 Response PDUs
    121487 Trap PDUs

Router#sh proc cpu | e 0.00
CPU utilization for five seconds: 88%/11%; one minute: 93%; five minutes: 93%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

151    95032220  69797440       1361 72.77% 77.21% 77.18%   0 SNMP ENGINE

thank you

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Michel Hegeraat Sat, 02/06/2010 - 06:42

If SNMP load is about the bring the router down you can be a bit more blunt.

See how many community strings are defined

Then limit the SNMP with an acl just to your monitoring system.

If the load is low reenable the other communities and management stations one by one using the acl's

Then you should notice the source causing the load.

Cheers,

Michel

Robert Ho Mon, 02/08/2010 - 00:08

there's only one snmp manager and one community with acl defined.

we even shut off the snmp manager and the router is still getting high cpu.

shouldnt the acl in the community filter all incoming requests?

i am wondering if it is snmp traffic coming from one of the public interfaces connecting the customers?

if so, how can we tell which interface without shutting down one at a time.

snmp-server community TESTING RO 99
!
access-list 99 remark SNMP Access List
access-list 99 permit 1.1.1.0 0.0.0.255
access-list 99 deny   any
!

Michel Hegeraat Mon, 02/08/2010 - 02:14

I there is just one community string and If you have removed the permit from the ACL and the SNMP still stays very high then I assume you have an IOS problem.

You can try to upgrade to the lastest IOS in the train you use right now. That should contain only fixes, no new features.

If that doesn't resolve I would open a TAC case.

Cheers,

Michel

jreekers Sun, 02/07/2010 - 05:17

Typically, this is due to a large table being polled (BGP table or routing table).

What you can do is run a "show stack ", in your case, "show stack 151".

The output can be decoded by TAC, and it will indicate which MIB object is the culprit.

From that information, you can configure an snmp "view" to exclude that object from being polled.

HTH,

-J

Robert Ho Mon, 02/08/2010 - 11:06

finally figured this thing out with the traditional command, show interface switching.

had to go through 100+ interfaces and check which one had the highest incremental CPU processed counter.

turns out that it was a customer on a their own vrf!

i am a bit shocked this is happening as there are no snmp configs defined for any customer vrf (we do not allow them to poll our box).

i've put in an acl for now to block it on the cust intf. what a relief.

Actions

This Discussion