Community Member

LMS 3.2 causing spikes on routers

Hi everyone,

A customer reported that their router experienced spikes (high cpu utilization) and claims that it is caused by snmp polling of the Ciscoworks server. What must be done to confirm that it is caused by snmp polling of LMS? If it is really caused by LMS, what values should be set on the polling parameters and threshold?

Thanks

Everyone's tags (2)
1 ACCEPTED SOLUTION

Accepted Solutions
Cisco Employee

Re: LMS 3.2 causing spikes on routers

Looks like the problem is related to VNM polling the MPLS-VPN-MIB.  If you are not using VNM, you can filter out this branch by applying an SNMP view to the community string used by LMS.  For example:

snmp-server view nomplsvpn iso included

snmp-server view nomplsvpn 1.3.6.1.3.118 excluded

snmp-server community mycomm view nomplsvpn RO

To fix this (bug CSCsl78965), you would need to upgrade to 12.2(18)SXF15 or higher.

15 REPLIES
Cisco Employee

Re: LMS 3.2 causing spikes on routers

When the CPU is spiking, you need to capture the output of "show stack PID" where PID is the process ID of the process which is taking up the CPU (probably either SNMP ENGINE or IP SNMP).  Post this stack trace along with the show version output.  This could be a known bug, or simply an issue with over-polling or polling a large amount of data.

Community Member

Re: LMS 3.2 causing spikes on routers

If its polling a large amount of data, what should be done to avoid spikes on the cpu? I'll be posting the output once I've received it.

Cisco Employee

Re: LMS 3.2 causing spikes on routers

The polling intervals can be changed, or the data can be spread out over multiple polls.

Community Member

Re: LMS 3.2 causing spikes on routers

Polling parameters and thresholds in DFM are still in its default values. What would be the ideal settings? Where else can we change the polling intervals?

Cisco Employee

Re: LMS 3.2 causing spikes on routers

It will depend on what part of LMS is causing the CPU spikes.

Community Member

Re: LMS 3.2 causing spikes on routers

How will I know which LMS process is causing the spikes? As I have mentioned, the polling parameters and thresholds are just in default value.

Cisco Employee

Re: LMS 3.2 causing spikes on routers

That's what the show stack output will indicate.  The objects being polled will indicate which part of LMS is doing the polling.

Community Member

Re: LMS 3.2 causing spikes on routers

Ok. This is noted. I will send the output once I receive it from the client. We have no access on their routers

Community Member

Re: LMS 3.2 causing spikes on routers

Hi Joseph,

Please see attached output of show version and show stack. Client has reported that the spikes occur every 4 hours (0000H 0400H 0800H so on..)

VAL-IPB-RTR-003#sh  ver

Cisco Internetwork  Operating System Software

IOS (tm) s72033_rp  Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF4, RELEASE  SOFTWARE (fc1)

Technical Support:  http://www.cisco.com/techsupport

Copyright (c)  1986-2006 by cisco Systems, Inc.

Compiled Thu  23-Mar-06 19:38 by tinhuang

Image text-base:  0x40101040, data-base: 0x42DA8000

ROM: System  Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)

BOOTLDR: s72033_rp  Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF4, RELEASE  SOFTWARE (fc1)

VAL-IPB-RTR-003  uptime is 4 years, 3 weeks, 1 day, 11 hours, 17 minutes

Time since  VAL-IPB-RTR-003 switched to active is 4 years, 3 weeks, 1 day, 11 hours, 34  minutes

System returned to  ROM by unknown reload cause - suspect boot_data[BOOT_COUNT] 0x0, BOOT_COUNT 0,  BOOTDATA 19 (SP by power on)

System restarted at  04:22:14 GMT Wed Jun 28 2006

System image file  is "disk1:s72033-adventerprisek9_wan-mz.122-18.SXF4.bin"

This product  contains cryptographic features and is subject to United

States and local  country laws governing import, export, transfer and

use. Delivery of  Cisco cryptographic products does not imply

third-party  authority to import, export, distribute or use encryption.

Importers,  exporters, distributors and users are responsible for

compliance with  U.S. and local country laws. By using this product you

agree to comply  with applicable laws and regulations. If you are unable

to comply with U.S.  and local laws, return this product immediately.

A summary of U.S.  laws governing Cisco cryptographic products may be found  at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require  further assistance please contact us by sending email to

export@cisco.com.

cisco CISCO7609  (R7000) processor (revision 1.1) with 983008K/65536K bytes of  memory.

Processor board ID  FOX0835049C

SR71000 CPU at  600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from  power-on

SuperLAT software  (copyright 1990 by Meridian Technology Corp).

X.25 software,  Version 3.0.0.

Bridging  software.

TN3270 Emulation  software.

39 Virtual  Ethernet/IEEE 802.3 interfaces

236 Gigabit  Ethernet/IEEE 802.3 interfaces

1917K bytes of  non-volatile configuration memory.

8192K bytes of  packet buffer memory.

65536K bytes of  Flash internal SIMM (Sector size 512K).

Configuration  register is 0x2102

VAL-IPB-RTR-003#

VAL-IPB-RTR-003# sh  proc cpu | i CPU

CPU  utilization for five seconds: 99%/1%; one minute: 99%; five minutes:  72%

220            0         2          0  0.00%  0.00%  0.00%   0 SLB Main CPU  pro

277       40076   25601858          1  0.00%  0.00%  0.00%   0 Max CPU  utilizat

VAL-IPB-RTR-003#sh  proc cpu | i SNMP

  29           16       107        149  0.00%  0.00%  0.00%   0 Cat6k  SNMP

  30         1244      1117       1113  0.00%  0.00%  0.00%   0 Cat6k SNMP  Trap

158            0       126          0  0.00%  0.00%  0.00%   0 SNMP  Timers

302            0         2          0  0.00%  0.00%  0.00%   0 EEM ED  SNMP

311    39506708  321176354        123  0.07%  0.08%  0.02%   0 IP SNMP

316   367797992 401044214        917 94.90% 30.84%  7.60%   0  SNMP ENGINE

317            0         1          0  0.00%  0.00%  0.00%   0 SNMP  ConfCopyPro

318     1266468     250441       5056  0.00%  0.00%  0.00%   0 SNMP Traps

VAL-IPB-RTR-003#sh  stack 316

Process 316:  SNMP  ENGINE

  Stack segment  0x52B6DC78 - 0x52B70B58

  FP: 0x52B70528,  RA: 0x40FE1228

  FP: 0x52B70550,  RA: 0x40934C50

  FP: 0x52B70578,  RA: 0x40934D80

  FP: 0x52B705B0,  RA: 0x40937DC4

  FP: 0x52B70608,  RA: 0x41FC8038

  FP: 0x52B70680,  RA: 0x41FC8C0C

  FP: 0x52B706F0,  RA: 0x41FC7084

  FP: 0x52B70710,  RA: 0x41FC6214

  FP: 0x52B70988,  RA: 0x40E4FDF8

  FP: 0x52B709D0,  RA: 0x40E300CC

  FP: 0x52B70A38,  RA: 0x40E23184

  FP: 0x52B70AB8,  RA: 0x40E477C0

  FP: 0x52B70B20,  RA: 0x40FD750C

  FP: 0x52B70B38,  RA: 0x40FD74F8

VAL-IPB-RTR-003#

Cisco Employee

Re: LMS 3.2 causing spikes on routers

Looks like the problem is related to VNM polling the MPLS-VPN-MIB.  If you are not using VNM, you can filter out this branch by applying an SNMP view to the community string used by LMS.  For example:

snmp-server view nomplsvpn iso included

snmp-server view nomplsvpn 1.3.6.1.3.118 excluded

snmp-server community mycomm view nomplsvpn RO

To fix this (bug CSCsl78965), you would need to upgrade to 12.2(18)SXF15 or higher.

Community Member

Re: LMS 3.2 causing spikes on routers

Thank you Sir.

Community Member

Re: LMS 3.2 causing spikes on routers

Hi Joseph,

Could you please tell us how did you manage to idenfiy that the problem is related to VNM Mib ? Is it through the show snmp stack command ? i

Thank you

Zied

Community Member

Re: LMS 3.2 causing spikes on routers

Hi  Joseph,

Just to explain we have exactly the same problem. Below is the show stack. Could this be related to the same VNM pollin

*Sep  5 12:02:43.230 GMT+1: %SYS-1-CPURISINGTHRESHOLD: Threshold: Total CPU Utilization(Total/Intr): 56%/14%, Top 3 processes(Pid/Util):  557/39%, 488/1%, 555/0%

*Sep  5 12:02:43.286 GMT+1: %HA_EM-4-LOG: CPUTH:

Process 557:  SNMP ENGINE

  Stack segment 0x1CFC204C - 0x1CFC4F2C

  FP: 0x1CFC49F0, RA: 0x9993528

  FP: 0x1CFC4A10, RA: 0x8C04F84

  FP: 0x1CFC4A20, RA: 0x8BEC170

  FP: 0x1CFC4A50, RA: 0xAB76E48

  FP: 0x1CFC4A58, RA: 0xAB773F8

  FP: 0x1CFC4A68, RA: 0xAB7776C

  FP: 0x1CFC4AA0, RA: 0xAE27A84

  FP: 0x1CFC4AB8, RA: 0xAFC75F4

  FP: 0x1CFC4AC0, RA: 0xA59A7AC

  FP: 0x1CFC4CF8, RA: 0xA5A38A8

  FP: 0x1CFC4D30, RA: 0xA59F194

  FP: 0x1CFC4DA0, RA: 0x973D4C0

  FP: 0x1CFC4E08, RA: 0x970BA00

  FP: 0x1CFC4E90, RA: 0x96F711C

  FP: 0x1CFC4F18, RA: 0x9729654

  FP: 0x1CFC4F20, RA: 0xAFAFF94

  FP: 0x0, RA: 0xAFAA270

Thank you

Cisco Employee

LMS 3.2 causing spikes on routers

Please start a new thread for your issue.  When you do, include the output of "show version".

Community Member

Re: LMS 3.2 causing spikes on routers

OK done

1615
Views
0
Helpful
15
Replies
CreatePlease to create content