cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
11219
Views
20
Helpful
12
Replies

high cpu from process hulc running con

ippolito
Level 1
Level 1

Strange problem on our 3750 stacks - when changing the config on many interfaces, the cpu utilization increases, to the point that sometimes my ssh connection to the switch is dropped, pings fail, etc.  E.g. if we do this command:

int range Fa1/0/1-48

  switchport access vlan 2

!

Happens consistently on all of our 3750 stacks.  Running 12.2(50)SE3 (ipbasek9).  Problem is worse if we run via script (i.e. send many commands one right after the other in rapid succession).

Thanks

Mike

12 Replies 12

Edison Ortiz
Hall of Fame
Hall of Fame

ippolito wrote:

Strange problem on our 3750 stacks - when changing the config on many interfaces, the cpu utilization increases, to the point that sometimes my ssh connection to the switch is dropped, pings fail, etc.  E.g. if we do this command:

int range Fa1/0/1-48

  switchport access vlan 2

!

Happens consistently on all of our 3750 stacks.  Running 12.2(50)SE3 (ipbasek9).  Problem is worse if we run via script (i.e. send many commands one right after the other in rapid succession).

Thanks

Mike

Normal behavior, please refer to this article:

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807213f5.shtml#topic4

With that said, your session shouldn't drop. The problem started after upgrading to 50SE3?


Regards

Edison.

Thanks for the reply -- the problem has been there even in previous software versions.

If ME3400? Is it also normal behavior?

Mike,

a 'sh processes cpu sorted' should show you the process that is reponsible for the CPU utilization. Can you post the output of that exec command ?

Load for five secs: 83%/28%; one minute: 54%; five minutes: 56%
Time source is NTP, 10:06:05.267 bkk Thu Sep 8 2016

CPU utilization for five seconds: 83%/28%; one minute: 54%; five minutes: 56%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 184   894281414 624659521       1431 17.22% 16.79% 16.06%   0 Spanning Tree    
 280    17922411   8387590       2136 14.56%  1.19%  0.30%   0 SNMP ENGINE      
 121    98529688 193780974        508  9.39%  7.23%  8.03%   0 Hulc LED Process
  71    11311244 169972899         66  4.07%  1.87%  1.94%   0 HLFM address lea
   4    28817921   2308991      12480  2.97%  0.40%  0.29%   0 Check heaps      

System image file is "flash:me340x-metroipaccessk9-mz.122-53.SE.bin"

cisco ME-3400G-12CS-A (PowerPC405) processor (revision C0) with 131072K bytes of memory.

Load for five secs: 99%/24%; one minute: 73%; five minutes: 67%
Time source is NTP, 12:18:40.984 bkk Thu Sep 8 2016

CPU utilization for five seconds: 99%/24%; one minute: 73%; five minutes: 67%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 276     4923536    391694      12569 39.83%  3.32%  0.69%   0 crypto sw pk pro
 184   898378679 625221974       1436 14.23% 15.65% 15.76%   0 Spanning Tree    
 121    99664001 194014411        513  9.11%  7.12%  7.86%   0 Hulc LED Process
  71    11785380 170188260         69  3.67%  3.28%  2.91%   0 HLFM address lea
 141    50818137   9203177       5521  0.95%  0.65%  0.64%   0 PI MATM Aging Pr
   4    28869836   2312567      12483  0.47%  0.29%  0.24%   0 Check heaps      
  86    19852030  78470032        252  0.47%  0.37%  0.37%   0 hpm main process
  58     8591248 300612692         28  0.15%  0.16%  0.15%   0 Fifo Error Detec
  89    12937781   6416237       2016  0.15%  0.17%  0.16%   0 hpm counter proc
 129     2725615   1417101       1923  0.15%  0.08%  0.06%   0 HQM Stack Proces
 102      510885  28670631         17  0.15%  0.05%  0.02%   0 Hulc Storm Contr

No any encryption process.

Each time differently.

Load for five secs: 100%/3%; one minute: 40%; five minutes: 35%
Time source is NTP, 10:34:23.777 bkk Fri Sep 9 2016

CPU utilization for five seconds: 100%/3%; one minute: 40%; five minutes: 35%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 184   931861303 632212539       1473 15.97% 17.17% 16.98%   0 Spanning Tree    
 217   4048289402886936966          0  5.27%  4.23%  4.22%   0 REP LSL Hello PP
 121   103338118 196447140        526  2.71%  1.90%  1.56%   0 Hulc LED Process
 141    51567399   9329846       5527  0.63%  0.61%  0.60%   0 PI MATM Aging Pr
  86    20265101  79609166        254  0.15%  0.33%  0.31%   0 hpm main process
  89    13128497   6503740       2018  0.15%  0.20%  0.21%   0 hpm counter proc
  35     1295404   5950482        217  0.15%  0.08%  0.09%   0 Per-Second Jobs  

The IOS we won't change cause services impact if it is stable.

The IOS we won't change cause services impact if it is stable.

INFORMATION REMOVED

uptime is 9 weeks, 5 days, 5 hours, 21 minutes

I found the spanning-tree instance is full 128 instances and some VLANs spanning-tree disable, we plan to change to WS-C4900M which support 4,094 instances.

Thank you very much.

Nice ... 4 points for helping out.  

You are on your own, mate.  :P

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: