Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

High CPU on 3560G switch

Hi Guys

I'm experiencing high CPU up to 100% frequently, can you advice how to find the cause?

-Sam

System image file is "flash:c3560-ipservicesk9-mz.122-58.SE2/c3560-ipservicesk9-mz.122-58.SE2.bin"

Switch Ports Model              SW Version            SW Image                

------ ----- -----              ----------            ----------              

*    1 52    WS-C3560G-48TS     12.2(58)SE2           C3560-IPSERVICESK9-M    

show proc cpu sort 5min

CPU utilization for five seconds: 58%/34%; one minute: 60%; five minutes: 77%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

  84    12001788     2278362       5267  6.40%  6.08%  6.69%   0 HLFM address lea

141    11412738     2821081       4045  2.24%  2.48%  4.59%   0 Hulc LED Process

142     6411109       74847      85657  2.71%  3.87%  3.70%   0 HL3U bkgrd proce

  71     2043009     4408879        463  1.91%  1.74%  1.23%   0 Fifo Error Detec

104     1349035      101712      13263  0.95%  0.77%  0.78%   0 hpm counter proc

  88      824936       10065      81960  0.00%  0.41%  0.46%   0 HULC Tcam Memory

153      531519       20145      26384  0.47%  0.34%  0.32%   0 HQM Stack Proces

212      381549      459073        831  0.31%  0.18%  0.29%   0 ADJ resolve proc

   4      438141       25503      17179  2.55%  0.35%  0.24%   0 Check heaps     

353      482354      154998       3112  0.00%  0.01%  0.17%   0 SNMP ENGINE     

show proc cpu his

      555555555555556666677777555555555555555555555555555555555555

      666677777777777777755555555555555577777888888888888888777775

  100                                                          

   90                                                          

   80                    *****                                 

   70               **********                                 

   60 **********************************************************

   50 **********************************************************

   40 **********************************************************

   30 **********************************************************

   20 **********************************************************

   10 **********************************************************

     0....5....1....1....2....2....3....3....4....4....5....5....6

               0    5    0    5    0    5    0    5    0    5    0

               CPU% per second (last 60 seconds)

      669999999999999999999988766667566696566766567665676666766667

      140210101109001011111299702326911100910220907229053041800305

  100            *                                             

   90   #####################*          *                      

   80   #####################**    *    *         *    *    *  

   70   ######################*    *    *    *    *    *    *  

   60 ##########################################################

   50 ##########################################################

   40 ##########################################################

   30 ##########################################################

   20 ##########################################################

   10 ##########################################################

     0....5....1....1....2....2....3....3....4....4....5....5....6

               0    5    0    5    0    5    0    5    0    5    0

               CPU% per minute (last 60 minutes)

              * = maximum CPU%   # = average CPU%

                          1                                                  

      9999999999999999989909999999                                           

      8648198885774573267909547892                                           

  100 ** * ******* **   ***** ***                                          

   90 ********##*####********#####                                         

   80 ********#######*****#**#####                                         

   70 *#*****########****#########                                         

   60 ############################                                         

   50 ############################                                         

   40 ############################                                         

   30 ############################                                         

   20 ############################                                         

   10 ############################                                         

     0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..

               0    5    0    5    0    5    0    5    0    5    0    5    0 

                   CPU% per hour (last 72 hours)

                  * = maximum CPU%   # = average CPU%

show controllers cpu-interface

ASIC    Rxbiterr   Rxunder    Fwdctfix   Txbuflos   Rxbufloc   Rxbufdrain

-------------------------------------------------------------------------

ASIC0     0          0          0          0          0          0        

ASIC1     0          0          0          0          0          0        

ASIC2     0          0          0          0          0          0        

ASIC3     0          0          0          0          0          0        

ASIC4     0          0          0          0          0          0        

ASIC5     0          0          0          0          0          0        

ASIC6     0          0          0          0          0          0        

ASIC7     0          0          0          0          0          0        

ASIC8     0          0          0          0          0          0        

ASIC9     0          0          0          0          0          0        

ASICA     0          0          0          0          0          0        

ASICB     0          0          0          0          0          0        

ASICC     0          0          0          0          0          0        

HOL Fix Counts

--------------

No Fixes:          0 Added:          1 In Use:       1063 Both:          0

CPU Heartbeat Statistics

Tx Success Tx Fail    1st Thr    2nd Thr    Unthr      RetryCtMax

---------- ---------- ---------- ---------- ---------- ----------

   1993464          0          8         25          8         86

Rx Delay

         0          1          2          3          4

---------- ---------- ---------- ---------- ----------

   1993062        225          0          0          0

AddlDelay AdvanceCnt

---------- ----------

         0          0

Rx Retries by RetryCount

         0          1          2          3          4          5          6

---------- ---------- ---------- ---------- ---------- ---------- ----------

   1992946          0          0          0         25         25         24

         7          8          9

---------- ---------- ----------

        22          0          0

AddlRetry

----------

       245

cpu-queue-frames  retrieved  dropped    invalid    hol-block  stray

----------------- ---------- ---------- ---------- ---------- ----------

rpc               1          0          0          0          0        

stp               50600      0          0          0          0        

ipc               0          0          0          0          0        

routing protocol  99762      0          0          0          0        

L2 protocol       23895      0          0          0          0        

remote console    0          0          0          0          0        

sw forwarding     199602629  0          0          0          0        

host              296387     0          0          0          0        

broadcast         130511     0          0          0          0        

cbt-to-spt        0          0          0          0          0        

igmp snooping     0          0          0          0          0        

icmp              96         0          0          0          0        

logging           0          0          0          0          0        

rpf-fail          0          0          0          0          0        

dstats            0          0          0          0          0        

cpu heartbeat     1993287    0          0          0          0        

cpu-queue         static inuse static added

----------------- ------------ ------------

rpc               0            0          

stp               0            0          

ipc               0            0          

routing protocol  0            0          

L2 protocol       0            0          

remote console    1063         1          

sw forwarding     0            0          

host              0            0          

broadcast         0            0          

cbt-to-spt        0            0          

igmp snooping     0            0          

icmp              0            0          

logging           0            0          

rpf-fail          0            0          

dstats            0            0          

cpu heartbeat     0            0

show platform port-asic stats drop

Port-asic Port Drop Statistics - Summary

========================================

  RxQueue 0 Drop Stats: 0

  RxQueue 1 Drop Stats: 0

  RxQueue 2 Drop Stats: 0

  RxQueue 3 Drop Stats: 0

  Port  0 TxQueue Drop Stats: 0

  Port  1 TxQueue Drop Stats: 0

  Port  2 TxQueue Drop Stats: 0

  Port  3 TxQueue Drop Stats: 0

  Supervisor TxQueue Drop Statistics

    Queue  0: 0

    Queue  1: 0

    Queue  2: 0

    Queue  3: 0

    Queue  4: 0

    Queue  5: 0

    Queue  6: 48144879

    Queue  7: 138

    Queue  8: 1001

    Queue  9: 0

    Queue 10: 0

    Queue 11: 0

    Queue 12: 0

    Queue 13: 0

    Queue 14: 0

    Queue 15: 177

Port-asic Port Drop Statistics - Details

========================================

  RxQueue Drop Statistics

    Queue 0

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 1

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 2

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

    Queue 3

    Weight 0 Frames: 0

    Weight 1 Frames: 0

    Weight 2 Frames: 0

  Port 0 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 1 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 2 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Port 3 TxQueue Drop Statistics

    Queue 0

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 1

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 2

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

    Queue 3

      Weight 0 Frames 0

      Weight 1 Frames 0

      Weight 2 Frames 0

  Supervisor TxQueue Drop Statistics

    Queue  0: 0

    Queue  1: 0

    Queue  2: 0

    Queue  3: 0

    Queue  4: 0

    Queue  5: 0

    Queue  6: 48144879

    Queue  7: 138

    Queue  8: 1001

    Queue  9: 0

    Queue 10: 0

    Queue 11: 0

    Queue 12: 0

    Queue 13: 0

    Queue 14: 0

    Queue 15: 177

show ip traff

IP statistics:

  Rcvd:  1126901 total, 326777 local destination

         0 format errors, 0 checksum errors, 27179 bad hop count

         0 unknown protocol, 0 not a gateway

         0 security failures, 0 bad options, 1 with options

  Opts:  0 end, 0 nop, 0 basic security, 0 loose source route

         0 timestamp, 0 extended security, 0 record route

         1 stream ID, 0 strict source route, 0 alert, 0 cipso, 0 ump

         0 other

  Frags: 0 reassembled, 0 timeouts, 0 couldn't reassemble

         0 fragmented, 0 couldn't fragment

  Bcast: 33173 received, 0 sent

  Mcast: 0 received, 0 sent

  Sent:  429529 generated, 1538467221 forwarded

  Drop:  1731 encapsulation failed, 0 unresolved, 0 no adjacency

         0 no route, 0 unicast RPF, 0 forced drop

         0 options denied, 0 source IP address zero

ICMP statistics:

  Rcvd: 5 format errors, 2 checksum errors, 2 redirects, 172 unreachable

        3508 echo, 5 echo reply, 0 mask requests, 0 mask replies, 0 quench

        0 parameter, 0 timestamp, 0 info request, 2 other

        0 irdp solicitations, 0 irdp advertisements

  Sent: 0 redirects, 104747 unreachable, 10 echo, 3508 echo reply

        0 mask requests, 0 mask replies, 0 quench, 0 timestamp

        0 info reply, 27179 time exceeded, 0 parameter problem

        0 irdp solicitations, 0 irdp advertisements

TCP statistics:

  Rcvd: 48365 total, 1 checksum errors, 52 no port

  Sent: 51143 total

UDP statistics:

  Rcvd: 274690 total, 0 checksum errors, 349 no port

  Sent: 242948 total, 0 forwarded broadcasts

BGP statistics:

  Rcvd: 0 total, 0 opens, 0 notifications, 0 updates

        0 keepalives, 0 route-refresh, 0 unrecognized

  Sent: 0 total, 0 opens, 0 notifications, 0 updates

        0 keepalives, 0 route-refresh

EIGRP-IPv4 statistics:

  Rcvd: 0 total

  Sent: 0 total

PIMv2 statistics: Sent/Received

  Total: 0/0, 0 checksum errors, 0 format errors

  Registers: 0/0 (0 non-rp, 0 non-sm-group), Register Stops: 0/0,  Hellos: 0/0

  Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0

  Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0

  Queue drops: 0

  State-Refresh: 0/0

IGMP statistics: Sent/Received

  Total: 0/0, Format errors: 0/0, Checksum errors: 0/0

  Host Queries: 0/0, Host Reports: 0/0, Host Leaves: 0/0

  DVMRP: 0/0, PIM: 0/0

  Queue drops: 0

OSPF statistics:

  Last clearing of OSPF traffic counters never

  Rcvd: 0 total, 0 checksum errors

        0 hello, 0 database desc, 0 link state req

        0 link state updates, 0 link state acks

  Sent: 0 total

        0 hello, 0 database desc, 0 link state req

        0 link state updates, 0 link state acks

ARP statistics:

  Rcvd: 99220 requests, 650 replies, 0 reverse, 0 other

  Sent: 478050 requests, 105 replies (0 proxy), 0 reverse

  Drop due to input queue full: 0

show platform tcam utilization

CAM Utilization for ASIC# 0                      Max            Used

                                             Masks/Values    Masks/values

Unicast mac addresses:                       1184/9472         32/141  

IPv4 IGMP groups + multicast routes:          152/1216          6/26   

IPv4 unicast routes:                            0/0             0/0    

IPv6 Multicast groups:                       1184/9472         32/141  

IPv6 unicast directly-connected routes:      1184/9472         32/141  

IPv6 unicast indirectly-connected routes:      24/192          12/43   

IPv4 policy based routing aces:                 0/0             0/0    

IPv4 qos aces:                                768/768         260/260  

IPv4 security aces:                          1024/1024         40/40   

IPv6 policy based routing aces:                 0/0             0/0    

IPv6 qos aces:                                  0/0             0/0    

IPv6 security aces:                           204/510           5/5    

Note: Allocation of TCAM entries per feature uses

a complex algorithm. The above information is meant

to provide an abstract view of the current TCAM utilization

show ip rou summ

IP routing table name is default (0x0)

IP routing table maximum-paths is 32

Route Source    Networks    Subnets     Replicates  Overhead    Memory (bytes)

connected       0           4           0           240         688

static          1           0           0           60          172

internal        2                                               932

Total           3           4           0           300         1792

  • LAN Switching and Routing
179
Views
0
Helpful
0
Replies
This widget could not be displayed.