I have found some interesting info during my troubleshooting. We are using a Dell Equalogic SAN. They have made an architectual reference document where they advise a small QoS config, looking like this:
switch(config)# mls qos
switch(config)# mls qos queue-set output 1 threshold 1 100 100 100 400
switch(config)# mls qos queue-set output 1 threshold 2 3200 100 10 3200
switch(config)# mls qos queue-set output 1 threshold 3 100 100 100 400
switch(config)# mls qos queue-set output 1 threshold 4 100 100 100 400
switch(config)# mls qos queue-set output 1 buffers 4 88 4 4
Basically this is some buffer tuning for output buffers. I have worked through this config very thouroughly and it does seem likely to help. I have found a document which describes a test case which proves that such buffer tuning should improve packet throughput about 25%.
What it does is allocating the highest percentage of buffers (88) to the ouput queue that is most utilized (infact the only queue utilized), int this case queue 2.
You will see if you do a show platform port-asic stats enqueue that most traffic will use queue 3. That is because default COS applied when mls qos is disabled is COS 0. When you enable mls qos, traffic will be assigned default COS 1. Based on this it will be directed to queue 2.
Also the max buffer threshold is configured to the highest value of 3200.
I am having a meeting this morning to discuss a test scenario, so I hope to update you soon with some results.
To see if you have the same issues do a sh platform port-asic stats drop
You should also not worry to much about traffic that is landing in other output queues, this is usually CPU traffic from the switch itself, including CDP, STP and HSRP.
Have you already got jumbo frames enabled?
