Overruns on a 6513

[markvincent]

We have a few ports that are showing about 200,000 overruns over a four day period. The ports are Gigabit ports connected to servers that have been having performance issues. Should we be concerned about the overruns, and does anyone know what might be causing them?

[Prashanth Krishnappa]

Cause for overruns

http://www.cisco.com/warp/public/473/53.shtml#overrun

What kind of line card and software are you running? Can you paste the output of sh mod from the switch?

[markvincent]

The software is Version 12.2(17d)SXB6.

Sh mod below:

Mod Ports Card Type Model Serial No.

--- ----- -------------------------------------- ------------------ -----------

5 48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX SAD074304WC

Mod MAC addresses Hw Fw Sw Status

--- ---------------------------------- ------ ------------ ------------ -------

5 000d.edb4.e790 to 000d.edb4.e7bf 6.0 7.2(1) 8.3(0.110)TE Ok

Mod Online Diag Status

--- -------------------

5 Pass

[Bobby Thekkekandam]

Overruns are the number of times the receiver hardware was unable to hand received data to a hardware buffer.

This is generally caused by the input rate of traffic exceeding the receiver's ability to handle the data.

What type of module is this? More than likely this is a blocking card who is receiving more data than can be put into the shared buffer. Depending on the type of card, we may be able to mitigate this by spreading the top talkers among different port groups.

[markvincent]

The module is a WS-X6548-GE-TX.

[Bobby Thekkekandam · Accepted solution]

This card shares a 1Mb buffer between groups ports (1-8, 9-16, 17-24, 25-32, 33-40, 41-48) since each block of eight ports is 8:1 oversubscribed. If any port in this range is receiving or transmitting traffic at a rate that exceeds its bandwidth or utilizing a large amount of buffers to handle bursts of traffic, the other ports in the same range of 8 may experience packet loss.

There are a couple of ways to address this.

1)Isolate any ports that may be consistently oversubscribed (SPAN destinations, servers with NFS, slower speed than the other ports in the range, etc.) to their own range of 8 ports to minimize the impact of drops to other interfaces.

2)Disable head of line blocking which will utilize the interface buffers instead of the shared buffers. This will result in only the single over utilized port having drops, but since the interface buffers (32k) are significantly smaller than the 1Mb shared buffer, there may be more lost traffic to individual ports. This is only recommended for extreme cases where slower clients or span ports cannot be moved to other line cards that offer dedicated interface buffers

6500(config)#service internal

6500(config)#interface gigabit 1/1

6500(config-if)#hol-blocking disable

%HOL Blocking is Disabled on: Gi1/1 Gi1/2 Gi1/3 Gi1/4 Gi1/5 Gi1/6 Gi1/7 Gi1/8

6500#show hol-blocking module 1

Interface Hol-Blocking

-------------- ---------------

Gi1/1 Disable

Gi1/2 Disable

Gi1/3 Disable

Gi1/4 Disable

Gi1/5 Disable

Gi1/6 Disable

Gi1/7 Disable

Gi1/8 Disable

Once this is disabled, the drops will move to the interface counters and can be seen with 'show interface gigabit '. The other ports will no longer be affected provided that they are not individually bursting also. Since it is recommended to keep hol-blocking enabled, this information can be used to find the device that is overrunning the buffers on the range of port and move it to another card or an isolated range on the card so hol-blocking can be re-enabled.

6500#show interface gigabit 1/1

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 25542147 <-----------

Additionally, this document may provide some more insight.

Buffers, Queues & Thresholds on Catalyst 6500 Ethernet Modules

http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper09186a0080131086.shtml

[markvincent]

Thanks, this should help.