I've originally posted this on Service Providers -> Metro forum but I was not sure if that was the right thing to do - so here I go:
We have several 3750MEs with the latest IOS 12.2(40)SE. We experience some annoying packet loss (around 1% or less) on any FastEthernet port no meter of its mode (we have almost all standard scenarios - trunk with sever vlans just passing the switch, access ports providing L3 vlan routing) when the traffic on a particular vlan on the port exceeds 20 Mbit/s.
We have checked and rechecked all cabling, configured all ports and machines connected to the switch manually to speed 100, duplex full, switchport nonegotiate etc.
However there is no problem when we do the same with any of the Gig ports (both normal and ES).
Switch resource monitors are as low as unplugged switch (cpu is under 10%, all tcam resources are below 10% too).
I've looked around the forum but didn't find any problem with great similarity to our.
So basically I'm totally out of ideas. Any help will be blessed.
P.S. For the record if I replace the 3750 with simple managed vlan switch (like HP ProCurve) there is no problem - I say this to eliminate questions about cabling and misconfiguration on attached ethernet intefaces.
Thank you for the diagnostic output. However, apart from the 3 input CRC errors, I don't see any evidence of packet lost. Where are you measuring it?
I'm bringing up vlan 11 interface with private ip address from the addressing space for the vlan and ping interfaces of devices connected at both ends of the vlan:
[Router A] -> Fast1/0/5 [3750me] ES1 -> HP Server
I don't have any packet loss between 3750 and the HP Server but have this annoying drops on the fast ethernet link.
Attaching ping stats and interface load. Note that current traffic on fast 1/0/5 is still low - things get far more worse with traffic growth, ending with 12% packet loss for traffic between 50-60 Mbit/s.
As I mentioned this is just one example on the network - we have the same issue with 3 more 3750me switches but on other 3 locations I've been forced to add simple vlan switches on the gig ports to avoid this because clients were very unhappy about it.
I will attach any information that is needed in order to debug this because I've totally ran out of ideas and can not believe that such an issue may exist on this kind of cisco device.
Thank you for your time helping me.
Ok, I'm giving up. Today will contact our hardware service provider with this issue. I seriously doubt that anyone else have the same problem so I'll just bet no the "faulty hardware" hand.
Today I have found out that shutting down MLS QOS (no mls qos) fixes the packet loss issue!
I've spoken to a friend who told me that one of his clients some time ago had the same issue with a 3650G and mentioned that he solved it by tuning queue-sets because when you enable mls qos it obeys certain default sets...
Anyone can give me a hand with this because I'm not so in with QoS in general, yet :)
Normally you use QOS in voip installations, if you don't have that then it is probably not even needed . If it was turned on it was probably working the way it was supposed to as regular data would only get so much bandwidth allocated and once it hits that threshold it would start dropping packets .
I need mls qos to be enabled in order to be able to attach policers on the ES ports. I like the feature to shape client vlans in both input and output, but it enforced default qos on 10/100 ports - and that was my problem.
Now I've tuned the buffers to different queue sets, mapped the traffic to the correct queues and everything is working fine.
Have you checked the version of IOS as there was an IOS bug on the 3750, the bug changed the queue depths which led to through put issues.
Bug id CSCeg29704 for details