MLS Rate Limitting Issue ?

[jdean1]

I have a strange issue. Our 6509/sup720 had some performance issues. Traffic traversing the switch was slow, ftp rates were horrible, etc. It didnt matter what servers and what ports were involved. I verified there were NO cabling/interface type issues/errors. When doing MTR's (traceroutes) I could see the higher level of pps that i tried to do the higher rate of dropped packets. I changed the IP unreachable MLS rate limit from 100 to 100000 and the issue went away. This is strange. What does MLS IP unreachable rate limits have to do with FTP and general file transfers? I can understand that it allows high rates of traceroutes, but why does this fix ALL traffic flowing through the switch? I tested this with a different 6509 and I can run the same level of MTR's through it without errors at the default of 100 ip unreachable rate limit. I would really like to understand this better.

Thanks.

[wdrootz]

The forwarding information base (FIB) rate-limiting allows all packets that require software processing to be rate limited. The following FIB rate-limiting usage guidelines apply:

1) FIB rate-limiting does not limit the rate of multicast traffic.

2) FIB rate-limiting does not differentiate between legitimate and illegitimate traffic (for example, tunnels, Telnet).

3) FIB rate-limiting applies aggregate rate-limiting and not per flow rate-limiting.

ARP throttling limits the rate at which packets destined to a connected network are forwarded to the route processor. Most of these packets are dropped, but a small number are sent to the router (rate limited). Because the rate-limiting mechanism allows a certain number of packets to be forwarded for software processing, you can view the packet drop statistics by entering NetFlow show commands from the CLI.

[johgill]

Rate-limiter discussions can get fairly complex, but my guess is you had some traffic being "punted" to CPU. This should never happen in a healthy 6500 under normal traffic. The hardware usually cannot increase delay to any noticeable extent, software processing must be happening somewhere.

There are many things that can cause this, but basically you may have had a few routes improperly programmed (or maybe you ran out of TCAM?) and traffic was routed incorrectly. If a situation like this is occurring, and an unreachable needs to be sent, that traffic is sent to the CPU and will be slowed down. Log statements in an ACL are also common culprits.

I would recommend looking at what traffic was hitting your CPU, any logs, number of routes, etc.

A TAC case would be a good place to get definitive answers for this situation in this switch.