cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2683
Views
0
Helpful
8
Replies

How to fix high CPU on 3650C

Rudy R.
Level 1
Level 1

If you are searching due to finding out your 3650C has high CPU, you can try these tricks:

* add storm control

* upgrade to c3560c-universalk9-tar.122-55.EX3.tar

  (EX2 has an Hulc LED Process bug)

* If you are in area 0 with OSPF, turn it into an NSSA and reduce your routes!

Honestly, I am not sure what fixed my switch, but it was using a lot of CPU and rebooting itself.  It appears stable today (I did all three changes at once).  Attached is a graph showing CPU usage over time on the switch.

Oh, and the switch did not like the using archive...

archive download-sw /overwrite tftp://1.2.3.4/

c3560c-universalk9-tar.122-55.EX3.tar

So I did a

# copy tftp://1.2.3.4/

c3560c405ex-universalk9-mz.122-55.EX3/c3560c405ex-universalk9-mz.122-55.EX3.bin flash:

then

(config)# boot system

flash:/c3560c405ex-universalk9-mz.122-55.EX3.bin

This was on WS-C3560CG-8PC-S  

Old OS: 12.2(55)EX2

New OS: 12.2(55)EX3

8 Replies 8

Leo Laohoo
Hall of Fame
Hall of Fame

Oh, and the switch did not like the using archive...

archive download-sw /overwrite tftp://1.2.3.4/c3560c-universalk9-tar.122-55.EX3.tar

Errrr ... I have >50 of these (WS-C3560CG-8PC) and they are all running 12.2(55)EX3 on a Layer 2. 

I upgraded a few (for testing) with the "archive download-sw /over /no-boot-set tftp://1.2.3.4/c3560c-universalk9-tar.122-55.EX3.tar" and I don't see any issues or "failure" of this command.


The rest of the 3560CG I've deployed had their IOS upgraded using Cisco ZeroTouch.  Again, I didn't see any issues.

No issues with the CPU either. 

Maybe it was just the EX2 that came on the device.  Made it through the day, and CPU was low.

# show processes cpu sorted

Showed Hulc LED Process at 10% <--- not normal for the blinking LED to use 10% of the CPU. 

Maybe it was just the EX2 that came on the device.

I forgot to mention that all my 3560CG were initially running EX2.  I use ZeroTouch to upgrade all of them to EX3 plus slap a configuration template on them.

Rudy R.
Level 1
Level 1

Ah!  Found another one... will just update IOS this time (already disabled unused SFP port) and see if that gets the CPU lower than 47%.

zyz-white#sh interfaces gigabitEthernet 0/1 | include rate
  Queueing strategy: fifo
  5 minute input rate 24731000 bits/sec, 2758 packets/sec
  5 minute output rate 2924000 bits/sec, 1538 packets/sec
zyz-white#sh ver | include EX                             
Cisco IOS Software, C3560C Software (C3560c405ex-UNIVERSALK9-M), Version 12.2(55)EX2, RELEASE SOFTWARE (fc1)
BOOTLDR: C3560C Boot Loader (C3560C-HBOOT-M) Version 12.2(55r)EX1, RELEASE SOFTWARE (fc1)
System image file is "flash:/c3560c405ex-universalk9-mz.122-55.EX2/c3560c405ex-universalk9-mz.122-55.EX2.bin"
*    1 10    WS-C3560CG-8TC-S   12.2(55)EX2           C3560c405ex-UNIVERSALK9-M
zyz-white#sh processes cpu sorted                         
CPU utilization for five seconds: 45%/31%; one minute: 47%; five minutes: 47%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
 135   652688748 130171328       5014  7.39%  7.15%  7.09%   0 Hulc LED Process 
  80   199796523 163931519       1218  0.79%  1.20%  1.12%   0 HLFM address lea 
 196    25710741  79047519        325  0.29%  0.33%  0.35%   0 Spanning Tree    
  66    11734741   2447998       4793  0.19%  0.15%  0.14%   0 Adjust Regions   
  10    11917385  50175794        237  0.19%  0.26%  0.23%   0 ARP Input        
  82     1004611 164714671          6  0.09%  0.03%  0.00%   0 HLFM address ret 
 182     2636107  11078867        237  0.09%  0.03%  0.00%   0 ADJ resolve proc 
 211     2997298   6116886        490  0.09%  0.01%  0.00%   0 PI MATM Aging Pr 
   8    16351509    102001     160308  0.00%  0.14%  0.16%   0 Licensing Auto U 
   7           0         2          0  0.00%  0.00%  0.00%   0 License Client N 

Updating OS didn't help that much.  Really, it took chaning the device to an NSSA. Looks like the 3560C is just barely a Layer 3 device...  Hopefully, this solves the packet loss we were seeing through the device.

Hey Rudy,

Just a stupid question:  Do you see the CPU utilization go down when you console/Telnet/SSH into the appliance?

Yes, i'm having the similar issue while i'm SSH the device utilization goes normal. 

Hello,

 

can you post the running config of your 3650 ? There are a few bugs related to specific features...

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: