cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
7547
Views
14
Helpful
16
Replies

3560G and 2950 switch trunk discard errors

hxmengmetro
Level 1
Level 1

Hi All,

I have two 3560G 24 port switches. Each of them connects to some 3560G or 2950 switches. Trunks between 3560G are set as 1000/full. Trunks between 3560G and 2950 are set as 100/full. show int status also shows the interface negotiation is 100/full for trunks between 3560G and 2950. The issue is I keep getting outdiscard errors in trunks between 3560G and 2950. At 2950 switches, I see Recv-errors too. I checked all the trunks traffic. They are totally not high. Only serveal mbps. Most time even lower than 1mbps.

I googled this kind of issue online. I see it could be possibly caused by high volume traffic higher than the capacity. But it appears the traffic there is not high enough to cause this kind of issue. Is there any possiblity that could cause this problem?

The below is 3560G trunk configuration for 2950 switch

interface GigabitEthernet0/10
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-122,124-4094
switchport mode trunk
speed 100
duplex full
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trust

the trunk configuration at 2950 switch:

interface FastEthernet0/24
switchport trunk allowed vlan 1-122,124-4094
speed 100
duplex full

Very simple configuration. I changed them to auto auto. no luck either. Still got errors.

Any help is appreciated.

Lou

16 Replies 16

Leo Laohoo
Hall of Fame
Hall of Fame

1.  What is the IOS of both of the 2950?
2.  Can you post the result of the commands "sh controller ethernet " of the uplinks of the 2950 and the 3560?

Thank you leolaohoo.

3560G Version: 12.2(46)SE

2950 Version: 12.1(22)EA12

Below is the interface at 3560G connecting to 2950:

sh controllers ethernet-controller gi0/2

     Transmit GigabitEthernet0/2              Receive
   1507087245 Bytes                       1298276667 Bytes
    472345779 Unicast frames               346359550 Unicast frames
    414879564 Multicast frames                209717 Multicast frames
     52301512 Broadcast frames                426790 Broadcast frames
            0 Too old frames              1213911633 Unicast bytes
            0 Deferred frames               37504918 Multicast bytes
            0 MTU exceeded frames           46859252 Broadcast bytes
            0 1 collision frames                   0 Alignment errors
            0 2 collision frames                   0 FCS errors
            0 3 collision frames                   0 Oversize frames
            0 4 collision frames                   0 Undersize frames
            0 5 collision frames                   0 Collision fragments
            0 6 collision frames
            0 7 collision frames           115016279 Minimum size frames
            0 8 collision frames           133028060 65 to 127 byte frames
            0 9 collision frames            51578957 128 to 255 byte frames
            0 10 collision frames           25469461 256 to 511 byte frames
            0 11 collision frames            3421682 512 to 1023 byte frames
            0 12 collision frames           18481618 1024 to 1518 byte frames
            0 13 collision frames                  0 Overrun frames
            0 14 collision frames                  0 Pause frames
            0 15 collision frames
            0 Excessive collisions                 0 Symbol error frames
            0 Late collisions                      0 Invalid frames, too large
            0 VLAN discard frames                  0 Valid frames, too large
            0 Excess defer frames                  0 Invalid frames, too small
     86244480 64 byte frames                       0 Valid frames, too small
    253257649 127 byte frames
    357339417 255 byte frames                      0 Too old frames
     82724044 511 byte frames                      0 Valid oversize frames
     44557358 1023 byte frames                     0 System FCS error frames
    115403880 1518 byte frames                     0 RxPortFifoFull drop frame
           27 Too large frames
            0 Good (1 coll) frames
            0 Good (>1 coll) frames

Below is the interface at 2950 connecting to 3560G:

sh controllers ethernet-controller fa0/24

  Transmit                                Receive
418447273 Bytes                       3239166918 Bytes
2648090429 Frames                        43881779 Frames
   2299300 Multicast frames                     0 FCS errors
   7353776 Broadcast frames            1906731398 Multicast frames
         0 Pause frames                 572094508 Broadcast frames
         0 Single defer frames                  0 Control frames
         0 Multiple defer frames                0 Pause frames
         0 1 collision frames                   0 Unknown opcode frames
         0 2-15 collisions                      0 Alignment errors
         0 Late collisions                      0 Length out of range
         0 Excessive collisions                 2 Symbol error frames
         0 Total collisions                     2 False carrier errors
         0 Control frames                       0 Valid frames, too small
         0 VLAN discard frames               1087 Valid frames, too large
         0 Too old frames                       2 Invalid frames, too small
      2275 Tagged frames                        0 Invalid frames, too large
         0 Aborted Tx frames                  869 Discarded frames

  Transmit and Receive
1436101576 Minimum size frames
2894033631 65 to 127 byte frames
594957480 128 to 255 byte frames
603382621 256 to 511 byte frames
295644824 512 to 1023 byte frames
1157138395 1024 to 1518 byte frames
   5679888 1519 to 1522 byte frames

Ganesh Hariharan
VIP Alumni
VIP Alumni

Hi All,

I have two 3560G 24 port switches. Each of them connects to some 3560G or 2950 switches. Trunks between 3560G are set as 1000/full. Trunks between 3560G and 2950 are set as 100/full. show int status also shows the interface negotiation is 100/full for trunks between 3560G and 2950. The issue is I keep getting outdiscard errors in trunks between 3560G and 2950. At 2950 switches, I see Recv-errors too. I checked all the trunks traffic. They are totally not high. Only serveal mbps. Most time even lower than 1mbps.

I googled this kind of issue online. I see it could be possibly caused by high volume traffic higher than the capacity. But it appears the traffic there is not high enough to cause this kind of issue. Is there any possiblity that could cause this problem?

The below is 3560G trunk configuration for 2950 switch

interface GigabitEthernet0/10
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1-122,124-4094
switchport mode trunk
speed 100
duplex full
srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trust

the trunk configuration at 2950 switch:

interface FastEthernet0/24
switchport trunk allowed vlan 1-122,124-4094
speed 100
duplex full

Hi,

If you get  large number of deferred frames, or Out-Discard (also referred to as Out-Lost on some platforms), it means that the switch's output buffers have filled up and the switch had to drop these packets. This can be a sign that this segment is run at an inferior speed and/or duplex, or there is too much traffic that goes through this port.

Check out the below link

http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a008015bfd6.shtml#outlost

Hope to Help !!

Ganesh.H

Remember to rate the helpful post

Actually I know that outdiscards mean but it looks strange to me that the traffic going through this trunk is only serveral Mbps. They can run up to 100Mbps. So why I have the overspeed problem?

Lou

Actually I know that outdiscards mean but it looks strange to me that the traffic going through this trunk is only serveral Mbps. They can run up to 100Mbps. So why I have the overspeed problem?

Lou

Hi Lou,

Another reason  for seeing Out-Discard is that the number of outbound packets chosen to be discarded even though no packet errors have been detected due to a low or no buffer space available on the switch ports. This could be caused due to some bursty traffic. Are there any servers that are used at the same time for a same function, like databcckup on a server or any such application.

Hope that Help !!

Ganesh.H

Thanks Ganesh.H,

The downstream 2950 switches only connect to the end user computer. No servers. But one thing look weired to me is all the trunk interfaces from one 3560G switch to the other 2950 switches have the outdiscard errors at the same time. Last Friday, I got 5 alarms for outdiscard errors at the same time for these trunk links. It sounds like a multicast traffic to all 2950 switches.The other Giga trunk links have no problem.

We do have a backup server running every day. But the backup server doesn't drag any traffic through those 2950 switches. Our last Friday backup start from 3:00PM. I got alarms at 3:40PM. If the backup thing caused this issue, why did it take 40 minutes to get alarm? I get alarm from OpenNMS system when the errors increase.

Does the backup server send out broadcast traffic when it do backup? I don't believe so.

Lou

Thanks Ganesh.H,

The downstream 2950 switches only connect to the end user computer. No servers. But one thing look weired to me is all the trunk interfaces from one 3560G switch to the other 2950 switches have the outdiscard errors at the same time. Last Friday, I got 5 alarms for outdiscard errors at the same time for these trunk links. It sounds like a multicast traffic to all 2950 switches.The other Giga trunk links have no problem.

We do have a backup server running every day. But the backup server doesn't drag any traffic through those 2950 switches. Our last Friday backup start from 3:00PM. I got alarms at 3:40PM. If the backup thing caused this issue, why did it take 40 minutes to get alarm? I get alarm from OpenNMS system when the errors increase.

Does the backup server send out broadcast traffic when it do backup? I don't believe so.

Lou

Hi Lou,

Yes you are rite backup servers never sends broadcast for taking backups they all works on well know TCP ports like legato backup works on TCP port 7937 and for veritas backup works on TCP port 1000.

Hope to Help !!

Ganesh.H

shailesh.h
Level 1
Level 1

It is worth to capture the traffic using any type of packet capturing tool like Ethereal for the trunk and monitor contineous that port with the help of SNMP to check whether bandwidth shoot up beyond set value..  This will give more insight view what type of traffic and utilization...

Thanks. I checked the traffic through those trunk links. and the traffic corresponding the errors time is really light only 1Mbps or 2Mbps. No bursty traffic from the graph opennms shows to me. I would like capture the traffic by mirroring the port in the switch. I will monitor these trunks close this week to see what pattern they will have.

hello  Hailu,

i have a similar problem on my access switches (3750). Also low utilization but OutDiscards. It seems to be connected some how with QoS configuration. If i check the hardware counters i can see that my packets get dropped within my Q2 (funny enough the numbering is different with this command)  at the threshold 0.

#sh platform port-asic stats drop fastEthernet 1/0/5

  Interface Fa1/0/5 TxQueue Drop Statistics
    Queue 0
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 1
      Weight 0 Frames 55408
      Weight 1 Frames 3
      Weight 2 Frames 0
    Queue 2
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0
    Queue 3
      Weight 0 Frames 0
      Weight 1 Frames 0
      Weight 2 Frames 0

We already though about tuning the buffers but first wait for some feedback from our supporter (working on the case) regarding this before we do this step.

Greets

David

Thanks David for your comments. I ran the same command at my 3560G and found all the outdiscarded packets got drop in Queue 3. Like below:

Interface Gi0/2 TxQueue Drop Statistics
   Queue 0
     Weight 0 Frames 0
     Weight 1 Frames 0
     Weight 2 Frames 0
   Queue 1
     Weight 0 Frames 0
     Weight 1 Frames 0
     Weight 2 Frames 0
   Queue 2
     Weight 0 Frames 0
     Weight 1 Frames 0
     Weight 2 Frames 0
   Queue 3
     Weight 0 Frames 0
     Weight 1 Frames 0
     Weight 2 Frames 6026


Your idea to tune the buffer maybe helpful on this. I need research about this. If you get response from your supporter, could you please let me know what you find? Thank you so much! I will continue to try different ways. Maybe queue configuration change.

Hailu

I guess it maybe related to qos setting for the interface. I have below auto qos configuration for the trunk interface:

srr-queue bandwidth share 10 10 60 20
queue-set 2
priority-queue out
mls qos trust cos
auto qos voip trust

Hello Hailu,

i think i have found the reason for this behaviour. We  found out that we have applications (e.g. webservices) which utilises the port at 100 % for a short period of time (up to 95 Mbit but just for a few miliseconds). The servers are connected via  Gig interfaces  the clients via Fastethernet. This is the moment where we see a few drops. In general this should be handled by 802.3x (flow control) feature which also would prevent loosing packets. Therefore we have QoS enabled it is not recommended to enable both features at the same time. Flow control would send pause frames where QoS cannot be guranteed.

I will move on this direction to see if there are any solutions to this.

all the very best


David

Thanks David for the information. I checked my switches. No flow control is enabled for these trunk interfaces. But I removed the QoS configuration for these trunk links since there is no IP phone connected to the 2950 switches. It doesn't have to be there.

After removing the QoS setup yesterday, I still see few outdiscard packets in these trunks. Both ends have no flow control configured. 2950 doesn't support input/output flow control. All outdiscarded packets happen in 3560G switches. output flow-control is not supported in 3560G.


So still need figure out how to avoid these errors.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: