Strange SNMP Problem

Unanswered Question
Jul 21st, 2008

I have multiple 4506 switches having a problem with snmp.

After upgrading to IOS cat4000-i5k91s-mz.122-25.EWA13.bin, these switches are having an issue being monitored via snmp.

It almost seems like the snmp agent is frozen. None of my monitoring software..i.e. CiscoWorks, HP OV, or any other MIB browser can access these devices via snmp.

I have IP connectivity between the Net Mgt. Server and these switches(I can ping all of them) and there are no ACLs.

I have tried renaming the community strings to ensure that they are correct and even tried stopping/starting the SNMP agent (via the no snmp-server command..and turing it back on), but still no success.

Also, out of the bunch that arent working, there are 3 4506 switches that "ARE" working with that same EWA13 IOS. I have compared the configs on the ones that are working and the ones that arent and they are all the same.

At this point, I am inclined to think that maybe there is a bug in this IOS, but not sure.

Any thoughts/suggestions???

Thanks in Advance.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Joe Clarke Mon, 07/21/2008 - 12:24

Enable debug snmp packet on one of the failing switches, and see what output you get on your terminal. While ICMP connectivity exists, there may be new ACLs that were put in place to block udp/161 traffic to these devices.

keyonj Mon, 07/21/2008 - 13:31

Thanks for the reply Joe.

I debugged snmp packets a little earlier and it was kind of weird. The debug is contantly spitting out that it is recieving UDP snmp packets from the Net MGT. server, but it is like the switch isnt doing anything with them. When I do a sh snmp, the 'snmp packets input' is not even incrementing. It is stuck on 4119210 and will not move from that number.

Joe Clarke Mon, 07/21/2008 - 13:35

Please post the output of "show stack PID" where PID is the PID of the IP SNMP and SNMP ENGINE processes. For example:

show stack 110

You can use show proc cpu to get the process IDs for these two processes.

keyonj Tue, 07/22/2008 - 07:43

Here is what I got from those show stack commands:

<--!! Output for 'Sh stack 60' !!-->

Process 60: IP SNMP

Stack segment 0x12510BC0 - 0x1251A030

FP: 0x12519FB0, RA: 0x10300ABC

FP: 0x1251A020, RA: 0x108005C0

FP: 0x1251A028, RA: 0x10311A34

FP: 0x0, RA: 0x10308368

<--!!Output of 'Sh stack 113'-->

Process 113: SNMP ENGINE

Stack segment 0x125E6FB0 - 0x125F0420

FP: 0x125F0288, RA: 0x10300ABC

FP: 0x125F02C0, RA: 0x10302D54

FP: 0x125F02F8, RA: 0x10BCA2F8

FP: 0x125F0330, RA: 0x10BCA6DC

FP: 0x125F0398, RA: 0x10B9EEC0

FP: 0x125F0410, RA: 0x10BC1C18

FP: 0x125F0418, RA: 0x10311A34

FP: 0x0, RA: 0x10308368

Joe Clarke Tue, 07/22/2008 - 08:24

Your SNMP ENGINE is wedged trying to send an authFail trap. Either that, or you captured this stack at the moment it was sending an authFail trap. If repeated show stacks on SNMP ENGINE look the same, and the switch is not being polled, then this process is most likely wedged.

I didn't see a bug to explain this, so you might be hitting something new. A reload should work around it, but you may want to contact the TAC for more troubleshooting.

keyonj Tue, 07/22/2008 - 09:55

Thanks for the help. I kind of figured it was hung. I was hoping there was a workaround without a reboot. I have alot of switches and it can be difficult getting down time. I guess the best solution is to push for the downtime and simply go to another IOS. Again thanks for your help!


This Discussion