01-08-2014 03:18 AM - edited 03-07-2019 05:26 PM
Hello Cisco World and happy new year!
I'm seeing some IfOutDiscards on my port channel between 6509 and Big IP F5 LTM load balancer These are not excessive but are incrementing, there are no problems reported from users or application teams within my company, however I want to know why these are appearing.
The F5 interfaces flow control is set as F5 default "Pause RX/TX"
QOS is NOT configured on the switch
WS-C6509-E
#sh mod 8
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
8 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX
show interfaces counters errors mod 8
Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize OutDiscards
Gi8/44 0 0 0 0 0 6932
Gi8/45 0 0 0 0 0 2947
#sh run int gi8/45
switchport
switchport access vlan 999
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport trunk allowed vlan 131,132,137,139,140,144-149,152,155,156,158-166
switchport trunk allowed vlan add 168-170,173,174,176,178,181,182,192-197
switchport trunk allowed vlan add 200-203,300,400
switchport mode trunk
logging event trunk-status
logging event bundle-status
no snmp trap link-status
spanning-tree portfast
channel-protocol lacp
channel-group 104 mode active
end
sh run int Po104
Building configuration...
Current configuration : 498 bytes
!
interface Port-channel104
description
switchport
switchport access vlan 999
switchport trunk encapsulation dot1q
switchport trunk native vlan 999
switchport trunk allowed vlan 131,132,137,139,140,144-149,152,155,156,158-166
switchport trunk allowed vlan add 168-170,173,174,176,178,181,182,192-197
switchport trunk allowed vlan add 200-203,300,400
switchport mode trunk
logging event trunk-status
logging event bundle-status
spanning-tree portfast trunk
end
I'm guessing these OutDisguards are due the switch port buffer being unable to processes these packets because they are burst packets. Is there anything you can actually do? I'm thinking of changing flow control settings or maybe going up to 10GB ports, currently 1GB/sec = 1MB/ms. Its seems to me real hard to pin point these errors, could it be the application?
Any ideas are welcome!
01-08-2014 03:20 AM
Please post the output to the following commands:
1. sh interface
2. sh counter interface
Sent from Cisco Technical Support Wii App
01-08-2014 03:24 AM
Hi Leo,
show int gi8/45
GigabitEthernet8/45 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0014.f212.383c (bia 0014.f212.383c)
Description: + ukcam00-152-f5-p1.2 ****
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 8/255, rxload 6/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT
input flow-control is off, output flow-control is off
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:37, output hang never
Last clearing of "show interface" counters 44w3d
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 2947
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 27384000 bits/sec, 6596 packets/sec
5 minute output rate 35124000 bits/sec, 8142 packets/sec
180327430375 packets input, 105547436134334 bytes, 0 no buffer
Received 896895 broadcasts (896895 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
174813418302 packets output, 106004071814675 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
show int gi8/46
GigabitEthernet8/46 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0014.f212.383d (bia 0014.f212.383d)
Description: + ukcam00-152-f5-mgmt ****
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseT
input flow-control is off, output flow-control is on
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:40, output hang never
Last clearing of "show interface" counters 44w3d
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 56000 bits/sec, 19 packets/sec
5 minute output rate 32000 bits/sec, 29 packets/sec
495852169 packets input, 227906519062 bytes, 0 no buffer
Received 15655 broadcasts (0 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
699746077 packets output, 111032329919 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
show counters interface gi8/45
64 bit counters:
0. rxHCTotalPkts = 204061683459
1. txHCTotalPkts = 206266009005
2. rxHCUnicastPkts = 204060746098
3. txHCUnicastPkts = 204276029087
4. rxHCMulticastPkts = 937281
5. txHCMulticastPkts = 1492369568
6. rxHCBroadcastPkts = 14
7. txHCBroadcastPkts = 497610359
8. rxHCOctets = 108601624334924
9. txHCOctets = 109723719767841
10. rxTxHCPkts64Octets = 1050607
11. rxTxHCPkts65to127Octets = 171869952815
12. rxTxHCPkts128to255Octets = 69589864097
13. rxTxHCPkts256to511Octets = 24907816728
14. rxTxHCpkts512to1023Octets = 35465418484
15. rxTxHCpkts1024to1518Octets = 18853492233
16. txHCTrunkFrames = 206263939097
17. rxHCTrunkFrames = 204060746179
18. rxHCDropEvents = 0
32 bit counters:
0. rxCRCAlignErrors = 0
1. rxUndersizedPkts = 0
2. rxOversizedPkts = 0
3. rxFragmentPkts = 0
4. rxJabbers = 0
5. txCollisions = 0
6. ifInErrors = 0
7. ifOutErrors = 0
8. ifInDiscards = 0
9. ifInUnknownProtos = 0
10. ifOutDiscards = 3841069
11. txDelayExceededDiscards = 0
12. txCRC = 0
13. linkChange = 23
14. wrongEncapFrames = 0
All Port Counters
1. InPackets = 204061631261
2. InOctets = 108601600839051
3. InUcastPkts = 204060693898
4. InMcastPkts = 937281
5. InBcastPkts = 14
6. OutPackets = 206265938925
7. OutOctets = 109723689299635
8. OutUcastPkts = 204275959344
9. OutMcastPkts = 1492369279
10. OutBcastPkts = 497610311
11. AlignErr = 0
12. FCSErr = 0
13. XmitErr = 0
14. RcvErr = 0
15. UnderSize = 0
16. SingleCol = 0
17. MultiCol = 0
18. LateCol = 0
19. ExcessiveCol = 0
20. CarrierSense = 0
21. Runts = 0
22. Giants = 0
23. InDiscards = 0
24. OutDiscards = 3841069
25. InErrors = 0
26. OutErrors = 0
27. InUnknownProtos = 0
28. txCRC = 0
29. TrunkFramesTx = 206263946312
30. TrunkFramesRx = 204060751665
31. WrongEncap = 0
32. Broadcast_suppression_discards = 0
33. Multicast_suppression_discards = 0
34. Unicast_suppression_discards = 0
35. rxTxHCPkts64Octets = 1050607
36. rxTxHCPkts65to127Octets = 171869957437
37. rxTxHCPkts128to255Octets = 69589867525
38. rxTxHCPkts256to511Octets = 24907817677
39. rxTxHCpkts512to1023Octets = 35465419496
40. rxTxHCpkts1024to1518Octets = 18853493869
41. DropEvents = 0
42. CRCAlignErrors = 0
43. UndersizedPkts = 0
44. OversizedPkts = 0
45. FragmentPkts = 0
46. Jabbers = 0
47. Collisions = 0
48. DelayExceededDiscards = 0
49. bpduOutlost = 0
50. qos0Outlost = 3841069
51. qos1Outlost = 0
52. qos2Outlost = 0
53. qos3Outlost = 0
54. qos4Outlost = 0
55. qos5Outlost = 0
56. qos6Outlost = 0
57. qos7Outlost = 0
58. qos8Outlost = 0
59. qos9Outlost = 0
60. qos10Outlost = 0
61. qos11Outlost = 0
62. qos12Outlost = 0
63. qos13Outlost = 0
64. qos14Outlost = 0
65. qos15Outlost = 0
66. qos16Outlost = 0
67. qos17Outlost = 0
68. qos18Outlost = 0
69. qos19Outlost = 0
70. qos20Outlost = 0
71. qos21Outlost = 0
72. qos22Outlost = 0
73. qos23Outlost = 0
74. qos24Outlost = 0
75. qos25Outlost = 0
76. qos26Outlost = 0
77. qos27Outlost = 0
78. qos28Outlost = 0
79. qos29Outlost = 0
80. qos30Outlost = 0
81. qos31Outlost = 0
82. bpduCbicOutlost = 0
83. qos0CbicOutlost = 0
84. qos1CbicOutlost = 0
85. qos2CbicOutlost = 0
86. qos3CbicOutlost = 0
87. bpduInlost = 0
88. qos0Inlost = 0
89. qos1Inlost = 0
90. qos2Inlost = 0
91. qos3Inlost = 0
92. qos4Inlost = 0
93. qos5Inlost = 0
94. qos6Inlost = 0
95. qos7Inlost = 0
96. qos8Inlost = 0
97. qos9Inlost = 0
98. qos10Inlost = 0
99. qos11Inlost = 0
100. qos12Inlost = 0
101. qos13Inlost = 0
102. qos14Inlost = 0
103. qos15Inlost = 0
104. qos16Inlost = 0
105. qos17Inlost = 0
106. qos18Inlost = 0
107. qos19Inlost = 0
108. qos20Inlost = 0
109. qos21Inlost = 0
110. qos22Inlost = 0
111. qos23Inlost = 0
112. qos24Inlost = 0
113. qos25Inlost = 0
114. qos26Inlost = 0
115. qos27Inlost = 0
116. qos28Inlost = 0
117. qos29Inlost = 0
118. qos30Inlost = 0
119. qos31Inlost = 0
120. pqueInlost = 0
121. Overruns = 0
122. maxIndex = 0
#show counters interface gi8/46
64 bit counters:
0. rxHCTotalPkts = 533905524
1. txHCTotalPkts = 761774290
2. rxHCUnicastPkts = 533885917
3. txHCUnicastPkts = 467174379
4. rxHCMulticastPkts = 144
5. txHCMulticastPkts = 285632202
6. rxHCBroadcastPkts = 19455
7. txHCBroadcastPkts = 8967719
8. rxHCOctets = 237665040716
9. txHCOctets = 128122576643
10. rxTxHCPkts64Octets = 41653443
11. rxTxHCPkts65to127Octets = 442419759
12. rxTxHCPkts128to255Octets = 661273707
13. rxTxHCPkts256to511Octets = 26779075
14. rxTxHCpkts512to1023Octets = 20848192
15. rxTxHCpkts1024to1518Octets = 102705646
16. txHCTrunkFrames = 8
17. rxHCTrunkFrames = 8
18. rxHCDropEvents = 0
32 bit counters:
0. rxCRCAlignErrors = 0
1. rxUndersizedPkts = 0
2. rxOversizedPkts = 0
3. rxFragmentPkts = 0
4. rxJabbers = 0
5. txCollisions = 0
6. ifInErrors = 0
7. ifOutErrors = 0
8. ifInDiscards = 0
9. ifInUnknownProtos = 0
10. ifOutDiscards = 0
11. txDelayExceededDiscards = 0
12. txCRC = 0
13. linkChange = 13
14. wrongEncapFrames = 0
All Port Counters
1. InPackets = 533905384
2. InOctets = 237664982447
3. InUcastPkts = 533885777
4. InMcastPkts = 144
5. InBcastPkts = 19455
6. OutPackets = 761774098
7. OutOctets = 128122551388
8. OutUcastPkts = 467174254
9. OutMcastPkts = 285632135
10. OutBcastPkts = 8967719
11. AlignErr = 0
12. FCSErr = 0
13. XmitErr = 0
14. RcvErr = 0
15. UnderSize = 0
16. SingleCol = 0
17. MultiCol = 0
18. LateCol = 0
19. ExcessiveCol = 0
20. CarrierSense = 0
21. Runts = 0
22. Giants = 0
23. InDiscards = 0
24. OutDiscards = 0
25. InErrors = 0
26. OutErrors = 0
27. InUnknownProtos = 0
28. txCRC = 0
29. TrunkFramesTx = 8
30. TrunkFramesRx = 8
31. WrongEncap = 0
32. Broadcast_suppression_discards = 0
33. Multicast_suppression_discards = 0
34. Unicast_suppression_discards = 0
35. rxTxHCPkts64Octets = 41653437
36. rxTxHCPkts65to127Octets = 442419605
37. rxTxHCPkts128to255Octets = 661273562
38. rxTxHCPkts256to511Octets = 26779075
39. rxTxHCpkts512to1023Octets = 20848188
40. rxTxHCpkts1024to1518Octets = 102705623
41. DropEvents = 0
42. CRCAlignErrors = 0
43. UndersizedPkts = 0
44. OversizedPkts = 0
45. FragmentPkts = 0
46. Jabbers = 0
47. Collisions = 0
48. DelayExceededDiscards = 0
49. bpduOutlost = 0
50. qos0Outlost = 0
51. qos1Outlost = 0
52. qos2Outlost = 0
53. qos3Outlost = 0
54. qos4Outlost = 0
55. qos5Outlost = 0
56. qos6Outlost = 0
57. qos7Outlost = 0
58. qos8Outlost = 0
59. qos9Outlost = 0
60. qos10Outlost = 0
61. qos11Outlost = 0
62. qos12Outlost = 0
63. qos13Outlost = 0
64. qos14Outlost = 0
65. qos15Outlost = 0
66. qos16Outlost = 0
67. qos17Outlost = 0
68. qos18Outlost = 0
69. qos19Outlost = 0
70. qos20Outlost = 0
71. qos21Outlost = 0
72. qos22Outlost = 0
73. qos23Outlost = 0
74. qos24Outlost = 0
75. qos25Outlost = 0
76. qos26Outlost = 0
77. qos27Outlost = 0
78. qos28Outlost = 0
79. qos29Outlost = 0
80. qos30Outlost = 0
81. qos31Outlost = 0
82. bpduCbicOutlost = 0
83. qos0CbicOutlost = 0
84. qos1CbicOutlost = 0
85. qos2CbicOutlost = 0
86. qos3CbicOutlost = 0
87. bpduInlost = 0
88. qos0Inlost = 0
89. qos1Inlost = 0
90. qos2Inlost = 0
91. qos3Inlost = 0
92. qos4Inlost = 0
93. qos5Inlost = 0
94. qos6Inlost = 0
95. qos7Inlost = 0
96. qos8Inlost = 0
97. qos9Inlost = 0
98. qos10Inlost = 0
99. qos11Inlost = 0
100. qos12Inlost = 0
101. qos13Inlost = 0
102. qos14Inlost = 0
103. qos15Inlost = 0
104. qos16Inlost = 0
105. qos17Inlost = 0
106. qos18Inlost = 0
107. qos19Inlost = 0
108. qos20Inlost = 0
109. qos21Inlost = 0
110. qos22Inlost = 0
111. qos23Inlost = 0
112. qos24Inlost = 0
113. qos25Inlost = 0
114. qos26Inlost = 0
115. qos27Inlost = 0
116. qos28Inlost = 0
117. qos29Inlost = 0
118. qos30Inlost = 0
119. qos31Inlost = 0
120. pqueInlost = 0
121. Overruns = 0
122. maxIndex = 0
01-08-2014 02:18 PM
qos0Outlost = 3841069
Thanks for the outputs.
I'm no expert in QoS but Gi8/45 is dropping packets because QoS queue 0 is overflowing. There is no noticable traffic or errors with Gi8/46.
01-09-2014 01:36 AM
With these "Oulost" counters, I am thinking rather the ports sharing the same 2 x 20Gb ASIC fabric on the 48 port line is to move the ports on different cards.
Its actually 4 port channel x 1GB
8/45
8/46
9/19
9/41
9/19 and 9/41 have the same problem
We only have 3 cards though.
What do you think.
01-09-2014 01:42 AM
You are using 6748 line card. This line card is designed for medium speed servers. It's got good memory buffers and should be able to handle the incoming traffic.
I am suspecting this could be an IOS bug.
Like I said, I am no expert in the "black magic" of QoS.
01-09-2014 01:52 AM
Thanks, I'll log a TAC case and update this discussion when I get some feedback from Cisco
01-09-2014 01:59 AM
Just want to make sure that you have enabled etherchannel on the F5 side?
Can you post the 6500 command output of "sh etherchannel 104 summary"?
01-09-2014 02:09 AM
see attached F5 config
sh etherchannel 104 summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator
M - not in use, no aggregation due to minimum links not met
m - not in use, port not aggregated due to minimum links not met
u - unsuitable for bundling
d - default port
w - waiting to be aggregated
Number of channel-groups in use: 8
Number of aggregators: 8
Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
104 Po104(SU) LACP Gi8/44(P) Gi8/45(P) Gi9/19(P)
Gi9/41(P)
Last applied Hash Distribution Algorithm: Fixed
01-09-2014 02:11 AM
Thanks for the output and screenshot. Looks fine to me.
01-09-2014 02:14 AM
Thought I would show this as well, each port is configured the same
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide