queue drops/flushes

[niro]

I have an etherchannel interface from our 6509 core to our 3750 distribution stack. On the 6509 I'm seeing a lot of input queue drops/flushes. The input traffic on this interface is never that high (the etherchannel consists of 3 1GB Fiber ports), the input is usually around 10Mb. The output utilization is much higher (few hundred MB average). I can't figure out why I get these input drops? Is there a way for me to figure out what's dropping?

here is the output of sh int po 7, as you can see I have about 4316 drops and I just cleared the counters 5 minutes ago:

Port-channel7 is up, line protocol is up (connected)

Hardware is EtherChannel, address is 0019.0761.4400 (bia 0019.0761.4400)

Description: uplink to Access Switch

Internet address is 172.16.14.182/30

MTU 1500 bytes, BW 3000000 Kbit, DLY 10 usec,

reliability 255/255, txload 18/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 1000Mb/s

input flow-control is off, output flow-control is off

Members in this channel: Gi3/1 Gi3/2 Gi3/3

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:26:57

Input queue: 0/75/4316/4316 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 6360000 bits/sec, 4630 packets/sec

5 minute output rate 218869000 bits/sec, 36449 packets/sec

L2 Switched: ucast: 0 pkt, 0 bytes - mcast: 1352 pkt, 476020 bytes

L3 in Switched: ucast: 5959235 pkt, 687080935 bytes - mcast: 1567083 pkt, 6592

66520 bytes mcast

L3 out Switched: ucast: 10870586 pkt, 9840853132 bytes mcast: 47951720 pkt, 34

346138679 bytes

6491680 packets input, 1175831651 bytes, 0 no buffer

Received 1358434 broadcasts (1567182 IP multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast, 0 pause input

0 input packets with dribble condition detected

50268764 packets output, 37805235561 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

[gpulos]

You can be getting the input drops due to hardware problems (port) and/or cable problems.

First be sure your etherchannels are configured properly and assure that all links in the channel use the same ASIC.

If it's a cable problem, get new cables and try those if you can.

(if these are copper GB links, be sure you have at least CAT5-E; if still problems, try CAT-6)

(if these are self made cables, be sure they're terminated 100% correctly)

If you want to figure out what traffic is dropping, the best way would be to use a sniffer.

You can use the sniffer to identify the source traffic on the LAN/VLAN and look for reponses to that traffic.

(depending on the protocol used, ie: TCP, you'll likely see alot of TCP SYNs and/or fast-restarts due to lack of acknowledgements of the traffic that was dropped)

A good sniffer is Ethereal/WireShark which can be downloaded free at the following link:

http://ethereal.com/download.html

[niro]

The ports I'm using are on module 3, ports 1, 2 and 3. the module is WS-X6724-SFP. The etherchannel on the 3750 is set up with 1 fiber port per stack switch (3 switches in a stack, 1 port each). How can I tell if they use the same asic?

Also, the etherchannel is configure in mode on (channel-group 7 mode on) on both switches.

[niro]

I still haven't replaced any of the three cables but I'm wondering if could still be cable issue if there are no errors on any of the interfaces? I'm still getting these drops flushes, here's another output a few mins after I cleared the interfaces:

Input queue: 0/75/4145/4145 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 10523000 bits/sec, 7721 packets/sec

5 minute output rate 358238000 bits/sec, 51191 packets/sec

input is 10Mb, that's almost nothing...and there are no errors/throttles on the port channel or the interfaces.

[tc-nakajima]

Have you checked below URL?

Generally, Input queue drops happens when Input Queue is full.

There is troubleshooting steps so you may narrow down the symptom if you follow.

Troubleshooting Input Queue Drops and Output Queue Drops

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a0080094791.shtml

Flushes occurr when low priority packets are discarded.

Input queue

The flushes counter can be used but never increments on the Catalyst 4000 Series that run Cisco IOS.)

SPD is a mechanism that quickly drops low priority packets when the CPU is

overloaded in order to save some processing capacity for high priority packets.

The flushes counter in the show interface command output increments as part of

selective packet discard (SPD), which implements a selective packet drop policy

on the IP process queue of the router. Therefore, it applies to only process switched traffic.

http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a008015bfd6.shtml

Below URL is about SPD.

Understanding Selective Packet Discard (SPD)

http://www.cisco.com/en/US/products/hw/routers/ps167/products_tech_note09186a008012fb87.shtml

Hope this helps!

Taichi