Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

enable flow-control on ASR9K interface

Hi Group,

Could someone get me a clear understanding about flow-control feature enable on our ASR 9K device ? I tried to enable this feature under my interface, and commit "flow-control ingress/egress/bidirectional" command and "negotiation auto" command in placed already, and I reset  the interface by shutdown/no shutdown. But flow-control is still showing off.

Does it mean flow-control is only working for layer 2 interface type ?

Detail information below.

Thanks very much in advance  if you can share with your idea !

our end confguration  ASK9010 version 3.9.2

interface GigabitEthernet0/2/0/23

flow-control ingress

negotiation auto

transceiver permit pid all

!

interface GigabitEthernet0/2/0/23.429

ipv4 address 10.10.10.1 255.255.255.252

flow ipv4 monitor fmm1 sampler fsm1 ingress

encapsulation dot1q 429

!

peering end configuration ASK1002 XE OS

interface GigabitEthernet0/2/0

no ip address

no ip redirects

no ip unreachables

no ip proxy-arp

ip flow ingress

logging event link-status

logging event subif-link-status

load-interval 30

negotiation auto

ntp disable

no mop enabled

!

interface GigabitEthernet0/2/0.429

encapsulation dot1Q 429

ip address 10.10.10.2 255.255.255.252

ip access-group 115 in

no ip redirects

no ip unreachables

no ip proxy-arp

ntp disable

------ interface output from our end after reset interface

igabitEthernet0/2/0/23 is up, line protocol is up

  Interface state transitions: 25

  Hardware is GigabitEthernet, address is f866.f214.362f (bia f866.f214.362f)

   Internet address is Unknown

  MTU 1514 bytes, BW 1000000 Kbit

     reliability 255/255, txload 3/255, rxload 3/255

  Encapsulation ARPA,

  Full-duplex, 1000Mb/s, LXFDX, link type is autonegotiation

  output flow control is off, input flow control is off

  loopback not set,

  ARP type ARPA, ARP timeout 04:00:00

  Last input 00:00:00, output 00:00:00

  Last clearing of "show interface" counters 2d00h

  5 minute input rate 15512000 bits/sec, 7098 packets/sec

  5 minute output rate 15529000 bits/sec, 2132 packets/sec

     2309688790 packets input, 542130815304 bytes, 30871 total input drops

     30870 drops for unrecognized upper-level protocol

     Received 0 broadcast packets, 117943 multicast packets

              0 runts, 0 giants, 0 throttles, 0 parity

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

     3337749329 packets output, 4368379414104 bytes, 261493 total output drops

     Output 76 broadcast packets, 0 multicast packets

     0 output errors, 0 underruns, 0 applique, 0 resets

     0 output buffer failures, 0 output buffers swapped out

     4 carrier transitions

GigabitEthernet0/2/0/23.429 is up, line protocol is up

  Interface state transitions: 13

  Hardware is VLAN sub-interface(s), address is f866.f214.362f

  Internet address is 66.97.23.185/30

  MTU 1518 bytes, BW 1000000 Kbit

     reliability 246/255, txload 3/255, rxload 3/255

  Encapsulation 802.1Q Virtual LAN, VLAN Id 429,  loopback not set,

  ARP type ARPA, ARP timeout 04:00:00

  Last input 00:00:00, output 00:00:00

  Last clearing of "show interface" counters 2d00h

  5 minute input rate 15522000 bits/sec, 7131 packets/sec

  5 minute output rate 15487000 bits/sec, 2141 packets/sec

     2309573924 packets input, 542103669918 bytes, 87167 total input drops

     87069 drops for unrecognized upper-level protocol

     Received 0 broadcast packets, 87069 multicast packets

     3337893541 packets output, 4368493331355 bytes, 16 total output drops

     Output 0 broadcast packets, 0 multicast packets

9 REPLIES
Cisco Employee

enable flow-control on ASR9K interface

Flow control at the ethernet level is negotiated by the two end points.

It looks like your peer may not want to participate in this and therefore it is showing off.

You can verify what was negotiated with the show controller gig all command to see what happened down

at that autoneg level.

regards

xander

Xander Thuijs CCIE #6775 Principal Engineer ASR9000, CRS, NCS6000 & IOS-XR
New Member

Re: enable flow-control on ASR9K interface

Hi Alex,

Below is short output under my interface.and detail show controller gi 0/2/0/23 all in attachment. please check.

And also under my lab testing with two ASR 9010 routers, the flow-control feature could be able to enable both sides, but only when I disable "negotiation auto" under the interfaces.  So back to  in this case, my peering device is XE OS device, in these document,  there is not explicit command to enable flowcontrol feature under the interface level.  there is only saying that the flowcontrol could be able to enable by negotiation auto feature enable. if is this confiict  here ? as ASR 9K enable flow-control only when disable " negotiation auto" and XE 1000K router is only can be enable flow-control feature when it learnt from peering with "negotiation auto" is on.

so is this ture ? did I mis-understand here ? and what's negotiation auto feature can do ?

thanks, could we know ?

========================

sh controllers gi 0/2/0/23   

Mon Feb 17 14:43:33.997 EST

Operational data for interface GigabitEthernet0/2/0/23:

State:

    Administrative state: enabled

    Operational state: Up

    LED state: Green On

Media:

    Media type: X fiber over long-wl laser PMD, full duplex

    Optics:

        Vendor: CISCO-SUMITOMO

        Part number: SCP6G44-C1-BMH

        Serial number: SPC142501CQ

MAC address information:

    Operational address: f866.f214.362f

    Burnt-in address: f866.f214.362f

    No unicast addresses in filter

    No multicast addresses in filter

Autonegotiation enabled:

    Flow control restricted to: Bidirectional

Operational values:

    Speed: 1Gbps

    Duplex: Full Duplex

    Flowcontrol: None

    Loopback: None (or external)

    MTU: 1526

    MRU: 1526

    Inter-packet gap: standard (12)

Eric, thanks Again!

Cisco Employee

Re: enable flow-control on ASR9K interface

Thanks for that detail Eric.

I did some further research and here is the deal:

we won't need autonego in order for flow control to work. The PAUSE frame operation is aside from the autoneg.

From the output you provided, it shows that this end wants to do bi-directional flow control and that autoneg is enabled.

Because flow control is part of the autoneg, it means taht the remote side was not willing or capable of doing flow control and therefore the operational state is set to "None".

When you disable autoneg, and our flow control operaton is bidirectional as it is right now, we will pay attention to the PAUSE frames, and we will also send them. But if the remote side is not giving us any, then there is little value to that, or if we would send one, and remote is not paying attention to it, there is also little value add there.

One other thing I wanted to mention is that PAUSE frame operation used to be useful for those NIC's that had little buffering capabilities. These days that doesnt really hold true anymore so the value of Flow Control at this level is rather limited.

Not to mention the fact that most implementations are not priority aware (extensions to the flow control standard have expanded on this however).

If you are concerned with interface loads and congestion, then probably using MQC/QOS, shaping or policing is a better option to handle congestion or using WRED to avoid congestion.

regards

xander

Xander Thuijs CCIE #6775 Principal Engineer ASR9000, CRS, NCS6000 & IOS-XR
New Member

Re: enable flow-control on ASR9K interface

Thanks very much Alex for your quick response and taking much care of this!

It is clear right now. Since we have a lots of input drop happened under the interface level and XE device side interface  is showing over-subscription as well, and also there are lots of packet lost happened over this link when the traffic load is only reaching over  half of  this physical link limit .This is the reason, we try to enable flow-control feature between two routers.

Yes, enable QOS  etc feature is best practice under this case, but I really am not sure the end user streaming  expernecnie is still good or not  when I input traffic shaping on upstream end.

Do you have an idea with input drop and packet lost within this link ?

thanks !

Eric

Cisco Employee

Re: enable flow-control on ASR9K interface

I see Eric, thanks for that rationale.

So you have the a1k as a CE and the A9k as a PE in that model? and the A1K is reporting ingress traffic drops is that correct?

Yeah, only if those input drops are caused by buffer depletion on the ingress side (doubt that) is where pause frames come in sort of handy.

Input drops can come from anywhere right, the controller, the QFP/CPP etc.

So a few things to verify are:

1) show controller gig stat on the a1k to see if there are any low level driver drops such as frame errors, crc etc.

2) check the show interface counters and see if there were ingress drops because of queue misses

CPE#show int g0/1

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 90

size shows how many there are in the queue now, drops is what we dropped because of buffer drops, flushes is what we wanted to punt to the CPU but got flushed because of SPD (selective packet discard).

3) if you also have a default queue size of 75 as above example, set it to 4096 via:

int g0/

hold-queue 4096 in and out

4) if that is all no beef, then try to verify the interface accountable QFP drops via:

show platform hardware qfp etc commands (I think is the a1k equivalent of the XR operational command of:

show controller pse qfp stat drop

5) that must result in some hint or pointer already, it may be mcast or so or some proto that can't be routed/forwarded or acl dropped maybe that can account for the input drops.

regards

xander

Xander Thuijs CCIE #6775 Principal Engineer ASR9000, CRS, NCS6000 & IOS-XR
New Member

Re: enable flow-control on ASR9K interface

Yes, you are right .

I will ask this CE end A1K device for further checking.

And in this case, my end- PE interface is also showing input drop, this number is same as Mcast number. I already ask their tech to check why we can see lots of multicast packets coming in ( we are running eBGP over this link), but they have no clue about this.

223039136 packets input, 74243589052 bytes, 23656 total input drops

     23656 drops for unrecognized upper-level protocol

     Received 0 broadcast packets, 23656 multicast packets

by the way, why I got below alert when I issued below command on our/PE end? is this bug of version 3.9.2

sh controllers pse qfp interface gigabitEthernet 0/2/0/23 statistics drop-summary subinterfaces location 0/2/CPU0

ERROR: sysdb_datalist FAILED. ['sysdb' detected the 'warning' condition 'A SysDB client passed a path for an item where a directory path was expected, or vice versa, or a path containing incorrectly placed '/' or reserved characters']

many thanks! Eric

Cisco Employee

Re: enable flow-control on ASR9K interface

hi eric,

the command show controller pse qfp is for the SIP700 in XR

the command show platform hardware qfp is for the a1k

the command show controller np counters npX loc 0/Y/cpu0 is for ethernet cards in the a9k

to find out the hardware related drops.

but that is no longer necessary, the issue is clear!

you are receiving mcast packets that we have no purpose for.

ps XR 392 is very old and not a recommended release, I would urge you to move to XR 434.

the mcast handling is much better there.

this is not a bug per-se, but you appear to be receiving mcast packets that the hardware ends up punting to the CPU for state building and if you dont have multicast routing enabled then there is no value for this.

but other things could be the reception of ospf hellos or so or any 224.x.x.x traffic that ends up getting punted.

all not a problem.

but later releases have better LPTS filters to keep the stuff away from the CPU that we have no purpose for.

regards

xander

Xander Thuijs CCIE #6775
Principal Engineer 
ASR9000, CRS, NCS6000 & IOS-XR

Xander Thuijs CCIE #6775 Principal Engineer ASR9000, CRS, NCS6000 & IOS-XR
New Member

Re: enable flow-control on ASR9K interface

Hi Alex,

This is great help !

thanks a lot !

Eric

New Member

Re: enable flow-control on ASR9K interface

Hi Alex,

Our customer really appreciated your help on this case. Their other ISP link has packet lost as well currently They shared some interface output to us, could you mind please take a quick check to see if everything is abnormal here ?

By the way,  they are still working on identify multicast packet issue now.

Thanks very much again!

EXT-RT-18#sh controllers

TenGigabitEthernet0/0/0

     0 input vlan errors

     0 ingress over sub drops

TenGigabitEthernet0/1/0

     0 input vlan errors

     0 ingress over sub drops

GigabitEthernet0/2/0---------------------peering with our PE router

     161113 input vlan errors

     0 ingress over sub drops------------port-channel 100 is peering with other ISP link, packet lost happened as well.

GigabitEthernet0/2/1------------port-channel 10 is peering with other ISP link, packet lost happened as well.

     0 input vlan errors

     260 ingress over sub drops

GigabitEthernet0/2/2------------port-channel 10 is peering with other ISP link, packet lost happened as well.

     0 input vlan errors

     3220 ingress over sub drops

GigabitEthernet0/2/3------------port-channel 10 is peering with other ISP link, packet lost happened as well.

     0 input vlan errors

     1947 ingress over sub drops

GigabitEthernet0/2/4------------port-channel 10 is peering with other ISP link, pasket lost happened as well.

     0 input vlan errors

     323 ingress over sub drops

EXT-RT-18#sh controllers

TenGigabitEthernet0/0/0

     0 input vlan errors

     0 ingress over sub drops

TenGigabitEthernet0/1/0

     0 input vlan errors

     0 ingress over sub drops

GigabitEthernet0/2/0

     161203 input vlan errors

     0 ingress over sub drops

GigabitEthernet0/2/1

     0 input vlan errors

     260 ingress over sub drops

GigabitEthernet0/2/2

     0 input vlan errors

     3353 ingress over sub drops

GigabitEthernet0/2/3

     0 input vlan errors

     2150 ingress over sub drops

GigabitEthernet0/2/4

     0 input vlan errors

     323 ingress over sub drops

EXT-RT-18#show platform hardware qfp active statistics drop clear

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

Disabled                                     8921       9097269 

Icmp                                            6           627 

IpTtlExceeded                            11267382     657766064 

Ipv4Acl                                   3933578     785959704 

Ipv4AclLookupMiss                         3018998     558440351 

Ipv4Martian                              11521966     619768129 

Ipv4NoAdj                                 3139026     370058424 

Ipv4NoRoute                                 39646       3212618 

Ipv4Null0                                  215993      14715497 

MaxTu                                           7         10621 

ReassTimeout                                  186        224056 

TailDrop                                      898       1179567 

TcpBadfrag                                      1            60 

UidbNotCfgd                                     1            74 

UnconfiguredIpv4Fia                        751281     478895168 

UnconfiguredIpv6Fia                         53840       8035292 

EXT-RT-18#show platform hardware qfp active statistics drop clear

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                  91          4942 

Ipv4Acl                                         9           606 

Ipv4AclLookupMiss                               8           536 

Ipv4NoAdj                                       2           130 

EXT-RT-18#show platform hardware qfp active statistics drop clear

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                  17          1258 

Ipv4Acl                                         4           232 

Ipv4AclLookupMiss                               2           120 

Ipv4NoAdj                                       1            58 

EXT-RT-18#show platform hardware qfp active statistics drop

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                  20          1480 

Ipv4Acl                                        18         10519 

Ipv4AclLookupMiss                               6          1744 

Ipv4NoAdj                                       1            62 

EXT-RT-18#show platform hardware qfp active statistics drop

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                  32          2368 

Ipv4Acl                                        22         12433 

Ipv4AclLookupMiss                               8          1876 

Ipv4NoAdj                                       3          1222 

Ipv4Null0                                       4           216 

EXT-RT-18#show platform hardware qfp active statistics drop

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                  75          5018 

Ipv4Acl                                        34         13819 

Ipv4AclLookupMiss                              11          2062 

Ipv4NoAdj                                       6          1444 

Ipv4Null0                                       5           270 

after PING test

EXT-RT-18#show platform hardware qfp active statistics drop

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                 221         13133 

Ipv4Acl                                        68         16403 

Ipv4AclLookupMiss                              16          3752 

Ipv4NoAdj                                      13          2257 

Ipv4Null0                                       7           378 

EXT-RT-18#show platform hardware qfp active statistics drop

----------------------------------------------------------------

Global Drop Stats                         Packets        Octets 

---------------------------------------------------------------

IpTtlExceeded                                 458         27887 

Ipv4Acl                                       114         19335 

Ipv4AclLookupMiss                              20          3980 

Ipv4NoAdj                                      78          6142 

Ipv4Null0                                      38          2052 

EXT-RT-18#sh int g9   ig 0/2/0

GigabitEthernet0/2/0 is up, line protocol is up

  Hardware is SPA-5X1GE-V2, address is e804.6226.7520 (bia e804.6226.7520)

   MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 7/255, rxload 59/255

  Encapsulation 802.1Q Virtual LAN, Vlan ID  1., loopback not set

  Keepalive not supported

  Full Duplex, 1000Mbps, link type is auto, media type is LX

  output flow-control is off, input flow-control is off

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:08:00, output hang never

  Last clearing of "show interface" counters 3d18h

  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 233183000 bits/sec, 22438 packets/sec

  30 second output rate 29110000 bits/sec, 15756 packets/sec

     327256424 packets input, 256673085526 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 382812 multicast, 0 pause input

     515656431 packets output, 306162835882 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

EXT-RT-18#sh int gig 0/2/0   1

GigabitEthernet0/2/1 is up, line protocol is up

  Hardware is SPA-5X1GE-V2, address is e804.6226.75c9 (bia e804.6226.7521)

  Description: Cogent_Link1_CCT1016381

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 10/255, rxload 127/255

  Encapsulation ARPA, loopback not set

  Keepalive not supported

  Full Duplex, 1000Mbps, link type is auto, media type is T

  output flow-control is on, input flow-control is on

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:14, output 00:00:10, output hang never

  Last clearing of "show interface" counters 3d18h

  Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 501043000 bits/sec, 47246 packets/sec

  30 second output rate 40979000 bits/sec, 34409 packets/sec

     479375310 packets input, 562476897956 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 11741 multicast, 0 pause input

     331679915 packets output, 90168694249 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

Port-channel10 is up, line protocol is up

  Hardware is GEChannel, address is e804.6226.75c9 (bia e804.6226.75c9)

      MTU 1500 bytes, BW 4000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 12/255, rxload 135/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  ARP type: ARPA, ARP Timeout 04:00:00

    No. of active members in this channel: 4

        Member 0 : GigabitEthernet0/2/1 , Full-duplex, 1000Mb/s

        Member 1 : GigabitEthernet0/2/2 , Full-duplex, 1000Mb/s

        Member 2 : GigabitEthernet0/2/3 , Full-duplex, 1000Mb/s

        Member 3 : GigabitEthernet0/2/4 , Full-duplex, 1000Mb/s

    No. of PF_JUMBO supported members in this channel : 4

  Last input 00:11:57, output never, output hang never

  Last clearing of "show interface" counters 3d18h

  Input queue: 0/1500/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/160 (size/max)

  30 second input rate 2058667000 bits/sec, 194630 packets/sec

  30 second output rate 177352000 bits/sec, 143475 packets/sec

     2008030115 packets input, 2371096047343 bytes, 0 no buffer

     Received 0 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     1348271463 packets output, 359020829441 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

943
Views
0
Helpful
9
Replies
CreatePlease login to create content