03-05-2012 06:56 PM - edited 03-07-2019 05:21 AM
Hello,
I have been trying to troubleshoot an output drop issue on my Cat3750G-24TS switch and can't figure out what is causing output drops.
Any input from you guys would be very appreciated!
Here is some information from my switch:
Switch#show interfaces counters errors
Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize OutDiscards
Gi1/0/1 0 1 0 1 0 0
Gi1/0/2 0 1 0 1 0 9203
Gi1/0/3 0 0 0 0 0 0
Gi1/0/4 0 0 0 0 0 194431
Gi1/0/5 0 0 0 0 0 29443
Gi1/0/6 0 0 0 0 0 51055
Gi1/0/7 0 1 0 1 0 263160
Gi1/0/8 0 0 0 0 0 0
Gi1/0/9 0 0 0 0 0 932177
Gi1/0/10 0 0 0 0 0 0
Gi1/0/11 0 0 0 0 0 0
Gi1/0/12 0 0 0 0 0 0
Gi1/0/13 0 0 0 0 0 80123
Gi1/0/14 0 0 0 0 0 8541
Gi1/0/15 0 0 0 0 0 5011
Gi1/0/16 0 0 0 0 0 0
...
Port G1/0/13 is the most problematic interface, output drops counter keep going up at a rapid pace everyting time I do a show int (around 1% of packets are dropped) :
GigabitEthernet1/0/13 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0024.9856.688d (bia 0024.9856.688d)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 131/255, rxload 23/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters 00:33:20
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 86920
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 9317000 bits/sec, 2860 packets/sec
30 second output rate 51572000 bits/sec, 5940 packets/sec
5963933 packets input, 2489026935 bytes, 0 no buffer
Received 3 broadcasts (3 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 3 multicast, 0 pause input
0 input packets with dribble condition detected
12710888 packets output, 13628047061 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
The interface never goes overs 60% utilization so I don't see why I would get any drops:
switch# show controllers g1/0/13 utilization
Receive Bandwidth Percentage Utilization : 9
Transmit Bandwidth Percentage Utilization : 46
I've already tried upgrading the IOS to 12.2.55SE4 and i'm still getting those.
I'm sure im missing something...
Regards,
John
03-05-2012 07:02 PM
reliability 255/255, txload 131/255, rxload 23/255
Transmit Bandwidth Percentage Utilization : 46
That's about right. 51.37%.
You may want to enable QoS on the interface. Whatever is connected to Gi1/0/13 can't take the amount of data the switch is pushing (at 100 Mbps), hence the switch is dropping them.
03-05-2012 07:23 PM
Agree with Leo.
50% rate is just an average for 30 seconds. Traffic is never stable - so that can indeed go down in a certain moment - then that would mean that it can spike in the other one to sustain the rate of 50%. Those spike force the port to queue the triffic in the output buffer and if that is full to drop eventually. Those drops you see os out-discards.
QoS can help here as it not only allow you to spread traffic among the queues and give particular one a priority - but that laso allow you to fine tune your buffers depth also permitting those to take additional buffer from common pull.
Good Egress QoS explanation can be found here:
https://supportforums.cisco.com/docs/DOC-8093
Nik
03-05-2012 08:06 PM
Hi Jean-Francois Courteau,
03-06-2012 06:27 AM
Hello again,
Thanks for the replies!
I did some more testing on my switch and found out that disabling QOS on it stopped the output drops from hapenning completely.
I do run QOS with priority queuing on some interfaces that have VOIP traffic passing through.
Do you guys think I should increase the buffers?
I found this exemple in the link provided by Nik:
Switch#config term
Switch(config)#mls qos queue-set output 1 threshold 1 1000 1000 50 1000
Switch(config)#mls qos queue-set output 1 threshold 2 1000 1000 50 1000
Switch(config)#mls qos queue-set output 1 threshold 3 1000 1000 50 1000
Switch(config)#mls qos queue-set output 1 threshold 4 1000 1000 50 1000
Not sure what effect it will have on my 3750 switch?
Thanks!
John
03-06-2012 06:35 AM
You do not need any QoS on LAN switches, even if your run VoIP, and no matter what people of small experience will tell you.
As you noticed yourself, QoS only causes drops. Then tuning these values is another source of trouble.
Disable QoS, and you will be fine.
03-06-2012 07:53 PM
Hi Jean-Francois Courteau,
Yes, you may disable your Qos on the LAN. I believe the best practices when applying Qos is by internet level or external level not on the LAN.
-onin
03-06-2012 11:27 PM
Nice comment from Paolo
Though I need to say that i my practise I've seen both. Somietime QoS enabled cause the drops with buffers/queues not being properly tuned. And I agree fine tuning might be a pain and not clear at all. However default performance and buffer capacity of the 3750 switch is quite low and I saw many cases when the default parameters affected throughput heavily due to exausted buffers.
So buffer tuning can make a good trick.
John you may check
show mls qos int gi1/0/13
and
show queueing int gi1/0/13 to see whcih queue is dropping most.
Then you can try disabling QoS globally to see if that is giving improvement. If not or if you still need to have some particular traffic prioritized then you can enable QoS back and tune the buffers.
the values you found above for buffer tuning are possibly too much for begining as it will also cause the commong bufer to be exausted more quickly thus other ports may not be able to get from it. So just increase that slightly for the queue which drop and monitor.
nik
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide