Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

High CPU Usage / Dropped Packets - Switch Blade WS-CBS3120X-S

Hi all,

I have a couple of Switches Blade 3120, working as active-standby model (HSRP) on a new site deployment. There are other 20 sites more or less, working on the same model, without issues. But in this one, we are seeing a high cpu usage. The traffic going through the platform is 600Mbps (on peaks), and in this case we have 40% of CPU usage. Traffic should be close to 3 Gbps. When we tried to send the whole traffic through the platform, active switch began to drop packets on the majority of interfaces.

When we analyze the CPU usage, there is a special process called "HL3U bkgrd proce" always have the most CPU use, but we do not know what concerns. We do not know if it is caused because there are PBRs configured. It should not matter. How I mentioned, there are other sites working fine and have had always the same PBR number.

Could you guys help us?. Any idea what is causing the high usage?. Is there a special debug we could to perform to diagnose the issue?. Also, we have seen a high interrupt CPU usage (9% in this case).

Find attached the whole diagnosis outputs.

Thanks for your assistance guys.

Cheers,

Juan Pablo

bog-sib-INT-rtr-1#show processes cpu sorted 5sec

CPU utilization for five seconds: 30%/9%; one minute: 25%; five minutes: 23%

PID Runtime(ms)     Invoked      uSecs   5Sec   1Min   5Min TTY Process

157   140004809   107071220       1307 14.24% 10.19%  9.01%   0 HL3U bkgrd proce

119     6860957     1519183       4516  0.79%  0.59%  0.53%   0 hpm counter proc

166     2511492      302802       8294  0.15%  0.15%  0.15%   0 HQM Stack Proces

199     4182906    15255882        274  0.15%  0.21%  0.20%   0 IP Input        

357      237531      782101        303  0.15%  0.03%  0.00%   0 IP SNMP         

186         101         148        682  0.15%  0.09%  0.02%   1 Virtual Exec    

242       63071     2330717         27  0.15%  0.02%  0.00%   0 CEF: IPv4 proces

  12      163754      620353        263  0.15%  0.01%  0.00%   0 ARP Input       

   9           0           2          0  0.00%  0.00%  0.00%   0 License Client N

   8          41        1827         22  0.00%  0.00%  0.00%   0 WATCH_AFS       

  11          50           4      12500  0.00%  0.00%  0.00%   0 Image License br

   7           0           2          0  0.00%  0.00%  0.00%   0 Timers          

bog-sib-INT-rtr-1#sh ip cef summary

IPv4 CEF is enabled for distributed and running

VRF Default

119 prefixes (119/0 fwd/non-fwd)

Table id 0x0

Database epoch:        2 (119 entries at this epoch)

3 REPLIES
Hall of Fame Super Gold

High CPU Usage / Dropped Packets - Switch Blade WS-CBS3120X-S

Haven't played with this blade for a long time ...

1.  What IOS version do you have?

2.  Can you post the output to the command "sh interface "?

New Member

High CPU Usage / Dropped Packets - Switch Blade WS-CBS3120X-S

Hi Leolaohoo,

I had not played with this one too !!!!...

1). IOS version (It was recently updated)

----------------------------------------------------------

bog-sib-INT-rtr-1#sh ver

Cisco IOS Software, CBS31X0 Software (CBS31X0-UNIVERSALK9-M), Version 12.2(58)SE1, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Thu 05-May-11 04:08 by prod_rel_team

ROM: Bootstrap program is CBS31X0 boot loader

BOOTLDR: CBS31X0 Boot Loader (CBS31X0-HBOOT-M) Version 12.2(0.0.951)SE3, CISCO DEVELOPMENT TEST VERSION

bog-sib-INT-rtr-1 uptime is 2 weeks, 3 days, 17 hours, 14 minutes

System returned to ROM by power-on

System restarted at 00:59:27 UTC Sat Jun 9 2012

System image file is "flash:cbs31x0-universalk9-mz.122-58.SE1.bin"

2). What interface do you want to see?, do you want to see all interfaces? . This switch has 16 interfaces that connect servers, and other going to our client. Below, the state of the two kind of interfaces:

Interface to Client (Bearer)

--------------------------------------

TenGigabitEthernet1/0/1 is up, line protocol is up (connected)

  Hardware is Ten Gigabit Ethernet, address is 001f.275d.d81b (bia 001f.275d.d81b)

  Description: BearerNContent_Aggregrate

  MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 10/255, rxload 14/255

  Encapsulation ARPA, loopback not set

  Keepalive not set

  Full-duplex, 10Gb/s, link type is auto, media type is 10GBase-LR

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 2w3d, output hang never

  Last clearing of "show interface" counters 07:07:56

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 562469000 bits/sec, 83641 packets/sec

  5 minute output rate 430500000 bits/sec, 73141 packets/sec

     2020563158 packets input, 1739897855828 bytes, 0 no buffer

     Received 13257 broadcasts (13257 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 13257 multicast, 0 pause input

     0 input packets with dribble condition detected

     1745065310 packets output, 1347244137726 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

Interface to Server

--------------------------

GigabitEthernet1/0/8 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is 001f.275d.d808 (bia 001f.275d.d808)

  Description: bog-15

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 15/255, rxload 12/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output 00:00:17, output hang never

  Last clearing of "show interface" counters 07:09:12

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 19418

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 47705000 bits/sec, 7155 packets/sec

  5 minute output rate 58897000 bits/sec, 8011 packets/sec

     178178750 packets input, 153802177226 bytes, 0 no buffer

     Received 4091 broadcasts (0 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     212233312 packets output, 206621942776 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

Thanks for your help. I am losing my hair with this issue.

Cheers,

Juan P.

Hall of Fame Super Gold

High CPU Usage / Dropped Packets - Switch Blade WS-CBS3120X-S

Total output drops: 19418

Hmmmm ...

What's connected to GigabitEthernet1/0/8?  It sounds like the interface is pushing as much as it can but the remote end is can't process the inbound data and the switchport is dropping the packets.

893
Views
0
Helpful
3
Replies
CreatePlease to create content