cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5485
Views
0
Helpful
5
Replies

C3750E & Interface Buffer Pools

gnijs
Level 4
Level 4

All,

What do the buffers RxQFB, RxQ0, RxQ1, RxQ15 etc.. mean in the "sh buffers" output ?

Interface buffer pools:
RxQFB buffers, 2040 bytes (total 904, permanent 904):
     895 in free list (0 min, 904 max allowed)
     18497860 hits, 0 misses
RxQ0 buffers, 2040 bytes (total 1200, permanent 1200):
     700 in free list (0 min, 1200 max allowed)
     500 hits, 0 misses
RxQ1 buffers, 2040 bytes (total 128, permanent 128):
     6 in free list (0 min, 128 max allowed)
     126686331 hits, 3512849 fallbacks
RxQ2 buffers, 2040 bytes (total 128, permanent 128):
     0 in free list (0 min, 128 max allowed)
     128 hits, 0 fallbacks, 0 trims, 0 created
     0 failures (0 no memory)
RxQ3 buffers, 2040 bytes (total 128, permanent 128):
     1 in free list (0 min, 128 max allowed)
     418039249 hits, 3265942 fallbacks
RxQ4 buffers, 2040 bytes (total 128, permanent 128):
     1 in free list (0 min, 128 max allowed)
     6658523 hits, 52248 fallbacks
RxQ5 buffers, 2040 bytes (total 128, permanent 128):
     64 in free list (0 min, 128 max allowed)
     64 hits, 0 misses
RxQ6 buffers, 2040 bytes (total 128, permanent 128):
     0 in free list (0 min, 128 max allowed)
     128 hits, 0 misses
RxQ7 buffers, 2040 bytes (total 192, permanent 192):
     61 in free list (0 min, 192 max allowed)
     3370617 hits, 65 misses
RxQ8 buffers, 2040 bytes (total 64, permanent 64):
     0 in free list (0 min, 64 max allowed)
     94425576 hits, 91194669 misses
RxQ9 buffers, 2040 bytes (total 1, permanent 1):
     0 in free list (0 min, 1 max allowed)
     1 hits, 0 misses
RxQ10 buffers, 2040 bytes (total 64, permanent 64):
     1 in free list (0 min, 64 max allowed)
     370156615 hits, 11585433 fallbacks
RxQ11 buffers, 2040 bytes (total 16, permanent 16):
     0 in free list (0 min, 16 max allowed)
     16 hits, 0 misses
RxQ12 buffers, 2040 bytes (total 96, permanent 96):
     0 in free list (0 min, 96 max allowed)
     96 hits, 0 misses
RxQ13 buffers, 2040 bytes (total 16, permanent 16):
     0 in free list (0 min, 16 max allowed)
     16 hits, 0 misses
RxQ15 buffers, 2040 bytes (total 4, permanent 4):
     0 in free list (0 min, 4 max allowed)
     372062235 hits, 372062231 misses

IPC buffers, 2048 bytes (total 300, permanent 300):
     296 in free list (150 min, 500 max allowed)
     2494138 hits, 0 fallbacks, 0 trims, 0 created
     0 failures (0 no memory)
Jumbo buffers, 9240 bytes (total 200, permanent 200):
     200 in free list (0 min, 200 max allowed)
     0 hits, 0 misses

I have a switch where the RxQ15 buffer misses are increasing continiously and 0 are in the free list:

sh buffer | b RxQ15
RxQ15 buffers, 2040 bytes (total 4, permanent 4):
     0 in free list (0 min, 4 max allowed)
     372064342 hits, 372064338 misses

sh buffer | b RxQ15
RxQ15 buffers, 2040 bytes (total 4, permanent 4):
     0 in free list (0 min, 4 max allowed)
     372064358 hits, 372064354 misses

regards,

Geert

5 Replies 5

Ganesh Hariharan
VIP Alumni
VIP Alumni

Hi Geert,

CPU buffer pools are named as  RxQ0 to RxQ15 The switches have 16 different CPU queues for different types traffic each queue can only send a certain amount of traffic to the CPU. The port asic will drop the rest an overload on one queue should not cause problems
for the other queues.

To see what packets are located in the buffers use the


show buffer command

In your case RxQ15 is increasing the buffer means can you check the cpu utilization of the switch.

HTH

Regards

Ganesh.H

Hello Ganesh,

CPU utilization is very low on the switch:

sh process cpu hist
                                                             
    1111111                                                  
    1133333666666666688888666667777777777777777777766666777799
100                                                          
90                                                          
80                                                          
70                                                          
60                                                          
50                                                          
40                                                          
30                                                          
20                                                          
10 **********************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5   
               CPU% per second (last 60 seconds)
                                                             
    1  1  111 1 1 1 11 111 111   11    1 1322        11 111131
    0990992009090909009021911099900999909001999999999119605070
100                                                          
90                                                          
80                                                          
70                                                          
60                                                          
50                                                          
40                                                         *
30                                       * *               *
20                                       ***           * * *
10 ##########################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5   
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%

Looking into the buffer content of RxQ15 , i get:

(seems to be strange packets):

(seems to me that the buffer is used by different processes to communicate between each other maybe ?)

sh buffer pool RxQ15 packet

Buffer information for RxQ15 buffer at 0x4948A18
  data_area 0xF8154B0, refcount 1, next 0x4948DAC, flags 0x0
  linktype 0 (None), enctype 0 (None), encsize 0, rxtype 0
if_input 0x0 (None), if_output 0x0 (None)
  inputtime 00:00:00.000 (elapsed never)
  outputtime 00:00:00.000 (elapsed never), oqnumber 65535
  datagramstart 0xF8154DE, datagramsize 64, maximum size 2196
  mac_start 0x0, addr_start 0x0, info_start 0x0
  network_start 0xF815504, transport_start 0x0, caller_pc 0x806A20


       0:          FC0003F9 00000000 00804044      |..y......@D
      12: 00000028 00000000 00000000 F4000000  ...(........t...
      28: 00000000 00000000 00000000 00000000  ................
      44: 00000000 00000000 00000000 00000000  ................
      60: 00000000 00                          .....         

Buffer information for RxQ15 buffer at 0x4948DAC
  data_area 0xF815D70, refcount 1, next 0x0, flags 0x0
  linktype 0 (None), enctype 0 (None), encsize 0, rxtype 0
  if_input 0x0 (None), if_output 0x0 (None)
  inputtime 00:00:00.000 (elapsed never)
  outputtime 00:00:00.000 (elapsed never), oqnumber 65535
  datagramstart 0xF815D9E, datagramsize 64, maximum size 2196
  mac_start 0x0, addr_start 0x0, info_start 0x0
  network_start 0xF815DC4, transport_start 0x0, caller_pc 0x806A20


       0:          FC0003F9 00000000 00804044      |..y......@D
      12: 00000028 00000000 00000000 F5000000  ...(........u...
      28: 00000000 00000000 00000000 00000000  ................
      44: 00000000 00000000 00000000 00000000  ................
      60: 00000000 00                          .....         

Buffer information for RxQ15 buffer at 0x4949140
  data_area 0xF816630, refcount 1, next 0x49494D4, flags 0x0
  linktype 0 (None), enctype 0 (None), encsize 0, rxtype 0
  if_input 0x0 (None), if_output 0x0 (None)
  inputtime 00:00:00.000 (elapsed never)
  outputtime 00:00:00.000 (elapsed never), oqnumber 65535
  datagramstart 0xF81665E, datagramsize 64, maximum size 2196
  mac_start 0x0, addr_start 0x0, info_start 0x0
  network_start 0xF816684, transport_start 0x0, caller_pc 0x806A20


       0:          FC0003F9 00000000 00804044      |..y......@D
      12: 00000028 00000000 00000000 F2000000  ...(........r...
      28: 00000000 00000000 00000000 00000000  ................
      44: 00000000 00000000 00000000 00000000  ................
      60: 00000000 00                          .....         

Buffer information for RxQ15 buffer at 0x49494D4
  data_area 0xF816EF0, refcount 1, next 0x4948A18, flags 0x0
  linktype 0 (None), enctype 0 (None), encsize 0, rxtype 0
  if_input 0x0 (None), if_output 0x0 (None)
  inputtime 00:00:00.000 (elapsed never)
  outputtime 00:00:00.000 (elapsed never), oqnumber 65535
  datagramstart 0xF816F1E, datagramsize 64, maximum size 2196
  mac_start 0x0, addr_start 0x0, info_start 0x0
  network_start 0xF816F44, transport_start 0x0, caller_pc 0x806A20


       0:          FC0003F9 00000000 00804044      |..y......@D
      12: 00000028 00000000 00000000 F3000000  ...(........s...
      28: 00000000 00000000 00000000 00000000  ................
      44: 00000000 00000000 00000000 00000000  ................
      60: 00000000 00                          .....

Were you ever able to get to the bottom of this issue? We are seeing a similiar issue:

Interface buffer pools:

Syslog ED Pool buffers, 600 bytes (total 132, permanent 132):

     100 in free list (132 min, 132 max allowed)

     1224 hits, 0 misses

RxQFB buffers, 2040 bytes (total 904, permanent 904):

     898 in free list (0 min, 904 max allowed)

     2403548 hits, 0 misses

RxQ0 buffers, 2040 bytes (total 1200, permanent 1200):

     700 in free list (0 min, 1200 max allowed)

     500 hits, 0 misses

RxQ1 buffers, 2040 bytes (total 128, permanent 128):

     2 in free list (0 min, 128 max allowed)

     48276275 hits, 960284 fallbacks

RxQ2 buffers, 2040 bytes (total 128, permanent 128):

     0 in free list (0 min, 128 max allowed)

     128 hits, 0 fallbacks, 0 trims, 0 created

     0 failures (0 no memory)

RxQ3 buffers, 2040 bytes (total 128, permanent 128):

     2 in free list (0 min, 128 max allowed)

     41544795 hits, 821368 fallbacks

RxQ4 buffers, 2040 bytes (total 128, permanent 128):

     0 in free list (0 min, 128 max allowed)

     4602417 hits, 5797274 misses

RxQ5 buffers, 2040 bytes (total 128, permanent 128):

     64 in free list (0 min, 128 max allowed)

     64 hits, 0 misses

RxQ6 buffers, 2040 bytes (total 128, permanent 128):

     0 in free list (0 min, 128 max allowed)

     21950 hits, 22032 misses

RxQ7 buffers, 2040 bytes (total 192, permanent 192):

     63 in free list (0 min, 192 max allowed)

     5677427 hits, 0 misses

RxQ8 buffers, 2040 bytes (total 64, permanent 64):

     0 in free list (0 min, 64 max allowed)

     171643 hits, 187213 misses

RxQ9 buffers, 2040 bytes (total 1, permanent 1):

     0 in free list (0 min, 1 max allowed)

     1 hits, 0 misses

RxQ10 buffers, 2040 bytes (total 64, permanent 64):

     2 in free list (0 min, 64 max allowed)

     6928620 hits, 621650 fallbacks

RxQ11 buffers, 2040 bytes (total 16, permanent 16):

     0 in free list (0 min, 16 max allowed)

     267518 hits, 268190 misses

RxQ12 buffers, 2040 bytes (total 96, permanent 96):

     0 in free list (0 min, 96 max allowed)

     96 hits, 0 misses

RxQ13 buffers, 2040 bytes (total 16, permanent 16):

     0 in free list (0 min, 16 max allowed)

     16 hits, 0 misses

RxQ15 buffers, 2040 bytes (total 4, permanent 4):

     1 in free list (0 min, 4 max allowed)

     435591684 hits, 435591681 misses

IPC buffers, 2048 bytes (total 300, permanent 300):

     296 in free list (150 min, 500 max allowed)

     14481930 hits, 0 fallbacks, 0 trims, 0 created

     0 failures (0 no memory)

Jumbo buffers, 9240 bytes (total 200, permanent 200):

     200 in free list (0 min, 200 max allowed)

     0 hits, 0 misses

A number of our interfaces are dropping packets:

GigabitEthernet1/0/5 is up, line protocol is up (connected)

  MTU 1500 bytes, BW 15360 Kbit, DLY 100 usec,

     reliability 255/255, txload 24/255, rxload 30/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:04, output 00:00:04, output hang never

  Last clearing of "show interface" counters 1d01h

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 7354

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 1843000 bits/sec, 319 packets/sec

  30 second output rate 1477000 bits/sec, 312 packets/sec

     23196023 packets input, 8342864505 bytes, 0 no buffer

     Received 10538 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 10538 multicast, 0 pause input

     0 input packets with dribble condition detected

     28221743 packets output, 16252406649 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

I have had the same problem.... The fix for me was to disable QoS. Our application performance went down so far we could barely use our internet connection. Turning off QoS increased the packet buffers to one pool thus higher throughput.

In addition setup an etherchannel to the highly utilized up-links to other switches and to your servers. 

hi

I am seeing the similar issue.cpu utilization is very low. same symptom.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco