04-16-2010 11:49 AM - edited 03-07-2019 12:33 AM
Hi All,
I have two 3560G 24 port switches. Each of them connects to some 3560G or 2950 switches. Trunks between 3560G are set as 1000/full. Trunks between 3560G and 2950 are set as 100/full. show int status also shows the interface negotiation is 100/full for trunks between 3560G and 2950. The issue is I keep getting outdiscard errors in trunks between 3560G and 2950. At 2950 switches, I see Recv-errors too. I checked all the trunks traffic. They are totally not high. Only serveal mbps. Most time even lower than 1mbps.
I googled this kind of issue online. I see it could be possibly caused by high volume traffic higher than the capacity. But it appears the traffic there is not high enough to cause this kind of issue. Is there any possiblity that could cause this problem?
The below is 3560G trunk configuration for 2950 switch
interface GigabitEthernet0/10
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-122,124-4094
switchport mode trunk
speed 100
duplex full
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trust
the trunk configuration at 2950 switch:
interface FastEthernet0/24
switchport trunk allowed vlan 1-122,124-4094
speed 100
duplex full
Very simple configuration. I changed them to auto auto. no luck either. Still got errors.
Any help is appreciated.
Lou
04-16-2010 05:25 PM
1. What is the IOS of both of the 2950?
2. Can you post the result of the commands "sh controller ethernet
04-19-2010 05:46 AM
Thank you leolaohoo.
3560G Version: 12.2(46)SE
2950 Version: 12.1(22)EA12
Below is the interface at 3560G connecting to 2950:
sh controllers ethernet-controller gi0/2
Transmit GigabitEthernet0/2 Receive
1507087245 Bytes 1298276667 Bytes
472345779 Unicast frames 346359550 Unicast frames
414879564 Multicast frames 209717 Multicast frames
52301512 Broadcast frames 426790 Broadcast frames
0 Too old frames 1213911633 Unicast bytes
0 Deferred frames 37504918 Multicast bytes
0 MTU exceeded frames 46859252 Broadcast bytes
0 1 collision frames 0 Alignment errors
0 2 collision frames 0 FCS errors
0 3 collision frames 0 Oversize frames
0 4 collision frames 0 Undersize frames
0 5 collision frames 0 Collision fragments
0 6 collision frames
0 7 collision frames 115016279 Minimum size frames
0 8 collision frames 133028060 65 to 127 byte frames
0 9 collision frames 51578957 128 to 255 byte frames
0 10 collision frames 25469461 256 to 511 byte frames
0 11 collision frames 3421682 512 to 1023 byte frames
0 12 collision frames 18481618 1024 to 1518 byte frames
0 13 collision frames 0 Overrun frames
0 14 collision frames 0 Pause frames
0 15 collision frames
0 Excessive collisions 0 Symbol error frames
0 Late collisions 0 Invalid frames, too large
0 VLAN discard frames 0 Valid frames, too large
0 Excess defer frames 0 Invalid frames, too small
86244480 64 byte frames 0 Valid frames, too small
253257649 127 byte frames
357339417 255 byte frames 0 Too old frames
82724044 511 byte frames 0 Valid oversize frames
44557358 1023 byte frames 0 System FCS error frames
115403880 1518 byte frames 0 RxPortFifoFull drop frame
27 Too large frames
0 Good (1 coll) frames
0 Good (>1 coll) frames
Below is the interface at 2950 connecting to 3560G:
sh controllers ethernet-controller fa0/24
Transmit Receive
418447273 Bytes 3239166918 Bytes
2648090429 Frames 43881779 Frames
2299300 Multicast frames 0 FCS errors
7353776 Broadcast frames 1906731398 Multicast frames
0 Pause frames 572094508 Broadcast frames
0 Single defer frames 0 Control frames
0 Multiple defer frames 0 Pause frames
0 1 collision frames 0 Unknown opcode frames
0 2-15 collisions 0 Alignment errors
0 Late collisions 0 Length out of range
0 Excessive collisions 2 Symbol error frames
0 Total collisions 2 False carrier errors
0 Control frames 0 Valid frames, too small
0 VLAN discard frames 1087 Valid frames, too large
0 Too old frames 2 Invalid frames, too small
2275 Tagged frames 0 Invalid frames, too large
0 Aborted Tx frames 869 Discarded frames
Transmit and Receive
1436101576 Minimum size frames
2894033631 65 to 127 byte frames
594957480 128 to 255 byte frames
603382621 256 to 511 byte frames
295644824 512 to 1023 byte frames
1157138395 1024 to 1518 byte frames
5679888 1519 to 1522 byte frames
04-19-2010 06:23 AM
Hi All,
I have two 3560G 24 port switches. Each of them connects to some 3560G or 2950 switches. Trunks between 3560G are set as 1000/full. Trunks between 3560G and 2950 are set as 100/full. show int status also shows the interface negotiation is 100/full for trunks between 3560G and 2950. The issue is I keep getting outdiscard errors in trunks between 3560G and 2950. At 2950 switches, I see Recv-errors too. I checked all the trunks traffic. They are totally not high. Only serveal mbps. Most time even lower than 1mbps.
I googled this kind of issue online. I see it could be possibly caused by high volume traffic higher than the capacity. But it appears the traffic there is not high enough to cause this kind of issue. Is there any possiblity that could cause this problem?
The below is 3560G trunk configuration for 2950 switch
interface GigabitEthernet0/10
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-122,124-4094
switchport mode trunk
speed 100
duplex full
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trustthe trunk configuration at 2950 switch:
interface FastEthernet0/24
switchport trunk allowed vlan 1-122,124-4094
speed 100
duplex full
Hi,
If you get large number of deferred frames, or Out-Discard (also referred to as Out-Lost on some platforms), it means that the switch's output buffers have filled up and the switch had to drop these packets. This can be a sign that this segment is run at an inferior speed and/or duplex, or there is too much traffic that goes through this port.
Check out the below link
Hope to Help !!
Ganesh.H
Remember to rate the helpful post
04-19-2010 06:39 AM
Actually I know that outdiscards mean but it looks strange to me that the traffic going through this trunk is only serveral Mbps. They can run up to 100Mbps. So why I have the overspeed problem?
Lou
04-19-2010 07:13 AM
Actually I know that outdiscards mean but it looks strange to me that the traffic going through this trunk is only serveral Mbps. They can run up to 100Mbps. So why I have the overspeed problem?
Lou
Hi Lou,
Another reason for seeing Out-Discard is that the number of outbound packets chosen to be discarded even though no packet errors have been detected due to a low or no buffer space available on the switch ports. This could be caused due to some bursty traffic. Are there any servers that are used at the same time for a same function, like databcckup on a server or any such application.
Hope that Help !!
Ganesh.H
04-19-2010 07:39 AM
Thanks Ganesh.H,
The downstream 2950 switches only connect to the end user computer. No servers. But one thing look weired to me is all the trunk interfaces from one 3560G switch to the other 2950 switches have the outdiscard errors at the same time. Last Friday, I got 5 alarms for outdiscard errors at the same time for these trunk links. It sounds like a multicast traffic to all 2950 switches.The other Giga trunk links have no problem.
We do have a backup server running every day. But the backup server doesn't drag any traffic through those 2950 switches. Our last Friday backup start from 3:00PM. I got alarms at 3:40PM. If the backup thing caused this issue, why did it take 40 minutes to get alarm? I get alarm from OpenNMS system when the errors increase.
Does the backup server send out broadcast traffic when it do backup? I don't believe so.
Lou
04-19-2010 08:12 AM
Thanks Ganesh.H,
The downstream 2950 switches only connect to the end user computer. No servers. But one thing look weired to me is all the trunk interfaces from one 3560G switch to the other 2950 switches have the outdiscard errors at the same time. Last Friday, I got 5 alarms for outdiscard errors at the same time for these trunk links. It sounds like a multicast traffic to all 2950 switches.The other Giga trunk links have no problem.
We do have a backup server running every day. But the backup server doesn't drag any traffic through those 2950 switches. Our last Friday backup start from 3:00PM. I got alarms at 3:40PM. If the backup thing caused this issue, why did it take 40 minutes to get alarm? I get alarm from OpenNMS system when the errors increase.
Does the backup server send out broadcast traffic when it do backup? I don't believe so.
Lou
Hi Lou,
Yes you are rite backup servers never sends broadcast for taking backups they all works on well know TCP ports like legato backup works on TCP port 7937 and for veritas backup works on TCP port 1000.
Hope to Help !!
Ganesh.H
04-19-2010 08:23 AM
It is worth to capture the traffic using any type of packet capturing tool like Ethereal for the trunk and monitor contineous that port with the help of SNMP to check whether bandwidth shoot up beyond set value.. This will give more insight view what type of traffic and utilization...
04-19-2010 08:38 AM
Thanks. I checked the traffic through those trunk links. and the traffic corresponding the errors time is really light only 1Mbps or 2Mbps. No bursty traffic from the graph opennms shows to me. I would like capture the traffic by mirroring the port in the switch. I will monitor these trunks close this week to see what pattern they will have.
04-23-2010 07:00 AM
hello Hailu,
i have a similar problem on my access switches (3750). Also low utilization but OutDiscards. It seems to be connected some how with QoS configuration. If i check the hardware counters i can see that my packets get dropped within my Q2 (funny enough the numbering is different with this command) at the threshold 0.
#sh platform port-asic stats drop fastEthernet 1/0/5
Interface Fa1/0/5 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 55408
Weight 1 Frames 3
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
We already though about tuning the buffers but first wait for some feedback from our supporter (working on the case) regarding this before we do this step.
Greets
David
04-26-2010 06:13 AM
Thanks David for your comments. I ran the same command at my 3560G and found all the outdiscarded packets got drop in Queue 3. Like below:
Interface Gi0/2 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 6026
Your idea to tune the buffer maybe helpful on this. I need research about this. If you get response from your supporter, could you please let me know what you find? Thank you so much! I will continue to try different ways. Maybe queue configuration change.
Hailu
04-26-2010 09:08 AM
I guess it maybe related to qos setting for the interface. I have below auto qos configuration for the trunk interface:
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trust
05-04-2010 04:12 AM
Hello Hailu,
i think i have found the reason for this behaviour. We found out that we have applications (e.g. webservices) which utilises the port at 100 % for a short period of time (up to 95 Mbit but just for a few miliseconds). The servers are connected via Gig interfaces the clients via Fastethernet. This is the moment where we see a few drops. In general this should be handled by 802.3x (flow control) feature which also would prevent loosing packets. Therefore we have QoS enabled it is not recommended to enable both features at the same time. Flow control would send pause frames where QoS cannot be guranteed.
I will move on this direction to see if there are any solutions to this.
all the very best
David
05-06-2010 06:44 AM
Thanks David for the information. I checked my switches. No flow control is enabled for these trunk interfaces. But I removed the QoS configuration for these trunk links since there is no IP phone connected to the 2950 switches. It doesn't have to be there.
After removing the QoS setup yesterday, I still see few outdiscard packets in these trunks. Both ends have no flow control configured. 2950 doesn't support input/output flow control. All outdiscarded packets happen in 3560G switches. output flow-control is not supported in 3560G.
So still need figure out how to avoid these errors.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide