cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
670
Views
0
Helpful
3
Replies

Locate QoS dropped packets

jedellerby
Level 1
Level 1

I need some help to determine where packets are being dropped in the network for a new QoS configuration I've rolled out.

My test setup is running iperf from two source systems to a single target system. The two data flows are labelled at different priority levels and sure enough, my lower priority traffic gets dropped as expected .. so all is looking good. What I can't tell is where the data is getting dropped.

The data route traverses a 6509 (IOS), 4507 (IOS), 3750 all with QoS enabled, queueing setup and appropriate service policies or trust state applied.

I've looked at all the counters in the route and I can see that my traffic being dropped on the final leg by the queueing, so all good once again.

Cat4507 command - show interface counter mod x

Cat 6500 command - show counters interface x/x

Cat3750 commands - show platform pm if-numbers

show platform port-asic stats drop port x asic y

However, one thing that came to light is the mismatch of tx rate at one end of a port channel to the rx rate of the other end. Anybody got any idea why the two don't look the same? I cleared the counters at both end and left it running for 10+ minutes so expected the 5 minute rates to be the same:

CAT 6509

5 minute input rate 3722000 bits/sec, 4271 packets/sec

5 minute output rate 127230000 bits/sec, 11332 packets/sec

CAT 4507

5 minute input rate 96120000 bits/sec, 8553 packets/sec

5 minute output rate 2544000 bits/sec, 3263 packets/sec

How come I'm missing 30 odd Mbps of rx traffic on the 4507r? Are the algortithms different for calculating the rx rate on the platforms or something?

Thanks

Jed

3 Replies 3

jedellerby
Level 1
Level 1

If nobody can help with the above, how about this related question.

I can see packets are getting dropped from Q2 on my C3750 (12.2(20)SE4) whenever I try to load the interface even lightly.

FastEthernet1/0/26 is up, line protocol is up (connected)

Hardware is Fast Ethernet, address is 0013.6009.009e (bia 0013.6009.009e)

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

reliability 255/255, txload 17/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, media type is 10/100BaseTX

input flow-control is off, output flow-control is unsupported

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:49, output 00:00:01, output hang never

Last clearing of "show interface" counters 00:38:45

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 172000 bits/sec, 318 packets/sec

5 minute output rate 6996000 bits/sec, 619 packets/sec

792794 packets input, 56452222 bytes, 0 no buffer

Received 276 broadcasts (0 multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 61 multicast, 0 pause input

0 input packets with dribble condition detected

1566924 packets output, 2212696242 bytes, 0 underruns

0 output errors, 0 collisions, 2 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier, 0 PAUSE output

0 output buffer failures, 0 output buffers swapped out

sw01#sho mls qos int fa1/0/26 queu

FastEthernet1/0/26

Egress Priority Queue : enabled

Shaped queue weights (absolute) : 3 0 0 0

Shared queue weights : 1 70 25 5

The port bandwidth limit : 100

The port is mapped to qset : 1

sw01#sho mls qos int fa1/0/26 buffer

FastEthernet1/0/26

The port is mapped to qset : 1

The allocations between the queues are : 25 25 25 25

sw01#show platform port-asic stats drop port 3

...

Port 3 TxQueue Drop Statistics

Queue 0

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 0

Queue 1

Weight 0 Frames 347

Weight 1 Frames 0

Weight 2 Frames 0

Queue 2

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 29111

Queue 3

Weight 0 Frames 0

Weight 1 Frames 0

Weight 2 Frames 6527492

29111 DSCP 0 packets have been dropped, yet the Tx rate of the port is well below capacity so why is it I'm seeing drops at this low utilisation level. I'm getting an actual HTTP download of about 700kbps for this workstation, yet without the QoS it can push higher.

It seems that the buffering kicks in way before the link is heavily utilised which wasn't what I was expecting. Before I applied any of the QoS settings (derived from the SRND) we weren't dropping packets and throughput was much better.

Why am I seeing the buffering kick in so early?

Thanks

Jed

"Why am I seeing the buffering kick in so early? "

I suspect it might just be a poor or inadequate allocation of port resources when QoS is enabled. If just "poor", a later IOS might correct the issue as might manual QoS tuning.

As to manual tuning, the documentation I've seen, to me, is weak explaining both the amount of buffers actually available (I have in mind % references) and the optimal interplay between reserved and common pool buffers.

A high drop rate with low utilization often indicates insufficient egress queue capacity to handle expected traffic rate bursting. What's interesting about your posted stats, the W2 drops in queues 2 and 3, since this, I believe, indicates tail drop for the total size of the queue.

What you might try, since you have:

Shared queue weights : 1 70 25 5

but

The allocations between the queues are : 25 25 25 25

Try setting buffer allocations (somewhat) the inverse of your queue bandwidth allocations.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: