high cpu from process hulc running con

Unanswered Question
Nov 23rd, 2009

Strange problem on our 3750 stacks - when changing the config on many interfaces, the cpu utilization increases, to the point that sometimes my ssh connection to the switch is dropped, pings fail, etc.  E.g. if we do this command:

int range Fa1/0/1-48

  switchport access vlan 2

!

Happens consistently on all of our 3750 stacks.  Running 12.2(50)SE3 (ipbasek9).  Problem is worse if we run via script (i.e. send many commands one right after the other in rapid succession).

Thanks

Mike

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4 (5 ratings)
Loading.
Edison Ortiz Mon, 11/23/2009 - 15:12

ippolito wrote:

Strange problem on our 3750 stacks - when changing the config on many interfaces, the cpu utilization increases, to the point that sometimes my ssh connection to the switch is dropped, pings fail, etc.  E.g. if we do this command:

int range Fa1/0/1-48

  switchport access vlan 2

!

Happens consistently on all of our 3750 stacks.  Running 12.2(50)SE3 (ipbasek9).  Problem is worse if we run via script (i.e. send many commands one right after the other in rapid succession).

Thanks

Mike

Normal behavior, please refer to this article:

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00807213f5.shtml#topic4

With that said, your session shouldn't drop. The problem started after upgrading to 50SE3?


Regards

Edison.

ippolito Tue, 11/24/2009 - 06:53

Thanks for the reply -- the problem has been there even in previous software versions.

gpauwen Wed, 09/07/2016 - 04:12

Mike,

a 'sh processes cpu sorted' should show you the process that is reponsible for the CPU utilization. Can you post the output of that exec command ?

ปลาวาฬทราย RMUT... Wed, 09/07/2016 - 20:17

Load for five secs: 83%/28%; one minute: 54%; five minutes: 56%
Time source is NTP, 10:06:05.267 bkk Thu Sep 8 2016

CPU utilization for five seconds: 83%/28%; one minute: 54%; five minutes: 56%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 184   894281414 624659521       1431 17.22% 16.79% 16.06%   0 Spanning Tree    
 280    17922411   8387590       2136 14.56%  1.19%  0.30%   0 SNMP ENGINE      
 121    98529688 193780974        508  9.39%  7.23%  8.03%   0 Hulc LED Process
  71    11311244 169972899         66  4.07%  1.87%  1.94%   0 HLFM address lea
   4    28817921   2308991      12480  2.97%  0.40%  0.29%   0 Check heaps      

ปลาวาฬทราย RMUT... Wed, 09/07/2016 - 22:25

System image file is "flash:me340x-metroipaccessk9-mz.122-53.SE.bin"

cisco ME-3400G-12CS-A (PowerPC405) processor (revision C0) with 131072K bytes of memory.

Load for five secs: 99%/24%; one minute: 73%; five minutes: 67%
Time source is NTP, 12:18:40.984 bkk Thu Sep 8 2016

CPU utilization for five seconds: 99%/24%; one minute: 73%; five minutes: 67%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 276     4923536    391694      12569 39.83%  3.32%  0.69%   0 crypto sw pk pro
 184   898378679 625221974       1436 14.23% 15.65% 15.76%   0 Spanning Tree    
 121    99664001 194014411        513  9.11%  7.12%  7.86%   0 Hulc LED Process
  71    11785380 170188260         69  3.67%  3.28%  2.91%   0 HLFM address lea
 141    50818137   9203177       5521  0.95%  0.65%  0.64%   0 PI MATM Aging Pr
   4    28869836   2312567      12483  0.47%  0.29%  0.24%   0 Check heaps      
  86    19852030  78470032        252  0.47%  0.37%  0.37%   0 hpm main process
  58     8591248 300612692         28  0.15%  0.16%  0.15%   0 Fifo Error Detec
  89    12937781   6416237       2016  0.15%  0.17%  0.16%   0 hpm counter proc
 129     2725615   1417101       1923  0.15%  0.08%  0.06%   0 HQM Stack Proces
 102      510885  28670631         17  0.15%  0.05%  0.02%   0 Hulc Storm Contr

ปลาวาฬทราย RMUT... Fri, 09/09/2016 - 00:24

No any encryption process.

Each time differently.

Load for five secs: 100%/3%; one minute: 40%; five minutes: 35%
Time source is NTP, 10:34:23.777 bkk Fri Sep 9 2016

CPU utilization for five seconds: 100%/3%; one minute: 40%; five minutes: 35%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 184   931861303 632212539       1473 15.97% 17.17% 16.98%   0 Spanning Tree    
 217   4048289402886936966          0  5.27%  4.23%  4.22%   0 REP LSL Hello PP
 121   103338118 196447140        526  2.71%  1.90%  1.56%   0 Hulc LED Process
 141    51567399   9329846       5527  0.63%  0.61%  0.60%   0 PI MATM Aging Pr
  86    20265101  79609166        254  0.15%  0.33%  0.31%   0 hpm main process
  89    13128497   6503740       2018  0.15%  0.20%  0.21%   0 hpm counter proc
  35     1295404   5950482        217  0.15%  0.08%  0.09%   0 Per-Second Jobs  

The IOS we won't change cause services impact if it is stable.

Leo Laohoo Fri, 09/09/2016 - 06:02

The IOS we won't change cause services impact if it is stable.

INFORMATION REMOVED

ปลาวาฬทราย RMUT... Fri, 09/09/2016 - 01:44

uptime is 9 weeks, 5 days, 5 hours, 21 minutes

I found the spanning-tree instance is full 128 instances and some VLANs spanning-tree disable, we plan to change to WS-C4900M which support 4,094 instances.

Thank you very much.

Leo Laohoo Fri, 09/09/2016 - 06:01

Nice ... 4 points for helping out.  

You are on your own, mate.  :P

Actions

This Discussion