queues and qos

Answered Question
Jul 22nd, 2010
User Badges:
  • Bronze, 100 points or more

Hi every body.


I was reading about how qos help in congested network.  Let say we have a network  . Three hosts and a router is connected to ethernet. The link between hosts and router is not congested.  Thee is no congestion elsewhere in the network. a host sends a packet to router.  This router though configured with qos configurations receives the packet. Since there is no congestion, router will not prioritizing the traffic.  Will  received packet still get placed in some queue? or to put simply is there any queue associated with outgoing interface on router when no qos is configured on a router?


thanks have a good day.

Correct Answer by Giuseppe Larosa about 7 years 5 days ago

Hello Sarah,

you have found a good link by a well known author ( Pepelnjak is also author of books and writes in many places including his own blog iosinternals and on nanog forums)


>> The hardware queue shared between the CPU and the interface chipset cannot be reordered, as the CPU has no control over the operation of the interface hardware that might pick up a packet at any time. Once a packet enters the hardware queue, its position in the queue cannot be changed.


This is a clear description of my doubts: once the packet is in the HW queue is too late to provide any QoS differentiation.


For this reason the HW queue should get packets from the output of the SW queues also when congestion is not present it provides a way to give priority to small VoIP packets for example.


The HW queue size is minimized to only a few packets when SW Queues are enabled, this is reported also on some of those books that say that when the link is not full the SW queues are bypassed.


but if the HW queue is 4 packets "congestion" = 4 packets in the HQ queue or tx ring as it is called!


About your questions:

it is difficult to describe router internals:

packets are placed in a shared memory waiting to be transmitted the interface HW takes them from this memory and move them to the tx ring.

Actually in modern routers interrupts to main cpu are minimized by using the mechanism described above ( a packet memory)

The mentioned descriptors are internal tags added to the packet to allow low level and efficient moving of packets from packet memory to interfaces.


To be more practical: in my tests I usually had many traffic flows belonging to different traffic classes. To demonstrate good behavior of schedulers like CBWFQ  or MDDR the offered traffic had a total volume slightly more then outgoing interface speed (101% to 103%) but only one traffic class was over its limit. The expected result is that only packets of that exceeding class are dropped.

However, even using a load variable in time with the appropriate instruments results are in line with the idea that SW queues are not bypassed.



Have a nice weekend you too.


I' m more busy in this period so I'm less present in the forums




Hope to help

Giuseppe

Correct Answer by amahendi about 7 years 6 days ago

" Will  received packet still get placed in some queue? or to put simply is there any queue associated with outgoing interface on router when no qos is configured on a router?"


Yes, there is always a queue associated with every L3 interface on a router. This is the interface hold-queue in the outbound direction. For interfaces <= ~ 2 Mbps, the queueing defaults to fair-queue; over 2 Mbps, it's fifo.


The interface hold-queue, like the queues associated with a QoS policy, is a software queue.


As for the packet flow:


- CEF switched traffic: During the tx_interrupt, packets are placed on the tx_ring or the so-called 'hardware queue'. This is basically the buffers present on the PLIM. The tx_ring is a FIFO structure and it's size various according to HW / media.


- Process switched traffic: Process switched traffic, like traffic originated form the router itself, *always* go through the 'software' queues(irrespective of whether there is congestion or not), before being placed on the tx_ring.


- An interface getting 'congested' basically means, the tx_ring is overrun. When this happens, there is backpressure from the tx_ring so that CEF switched packets are queued in the software queued (intf hold-queue or QoS queues). Hence, any software queueing is invoked by this 'backpressure' mechanism. This mechanism itself varies depending on the hardware, it can either be the tx_ring getting overrun (this is the case for most lower-end platforms) or can be an internal shaper / ASIC that provides this mechanism (like in the case of SIP / SPA).


- Abhi

Correct Answer by Giuseppe Larosa about 7 years 6 days ago

Hello Sarah,


I think the SW queues are created and used before congestion happens otherwise how we could provide better QOS to some traffic classes?



My opinion is based on many tests I did of CBWFQ on C7500 and MDRR schedulers on 12000 routers using traffic generator instruments like SMARTBITS or Agilent router tester. The first instrument has the limit to be not able to change the traffic load during the test, but Agilent router tester can do this.


This is my opinion and this can be true only of these high end routers, however a show policy-map interface shows packets for each traffic class on an interface where CBWFQ is applied outbound also if the link is not congested.


In real networks congestion happens without any advice what about the packets already in the FIFO HW queue ?


I agree that the HW queue is still there but minimized in size (expressed as a number of packets) and that may be a sort of threshold mechanism might be used to bypass the SW queues when average load is low. But I don't know how this could handle the sudden change to congestion state, when the thr threshold is reached we start to send packets to SW queues instead of sending them directly to the HW queue.



Hope to help

Giuseppe

Correct Answer by Mohamed Sobair about 7 years 1 week ago

Hi Sarah,


You have hardware Queue and a software queue. The Hardware Queue is always FIFO where the software Queue is a queuing mechanism configurable by the user.


In your case, No Queing is configured, The hardware queue recieves tha packet and When a queue packet is full, Cisco IOS software has no place to put extra packet. The packet will be tail droped.



HTH

Mohamed

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (5 ratings)
Loading.
Correct Answer
Mohamed Sobair Thu, 07/22/2010 - 12:33
User Badges:
  • Gold, 750 points or more

Hi Sarah,


You have hardware Queue and a software queue. The Hardware Queue is always FIFO where the software Queue is a queuing mechanism configurable by the user.


In your case, No Queing is configured, The hardware queue recieves tha packet and When a queue packet is full, Cisco IOS software has no place to put extra packet. The packet will be tail droped.



HTH

Mohamed

Tharak Abraham Thu, 07/22/2010 - 18:22
User Badges:
  • Bronze, 100 points or more

The answer for your question is NO.

The packet would be transmitted without any queueing !

Why ?

Remember that the defined queues or the so called fancy queueing kicks in only when there is congestion.

eg: If the link is a T1, till the traffic crosses the link capacity of 1.54Mbps (theoretically)


Once when it crosses, the software queues kicks in and queues your traffic as per the defined conditions say PQ, WFQ et al

The Scheduler picks it up from the queues and puts it into the TxRing and eventually on to the medium.


There are some default standards defined in Cisco, any link lesser than E1 capacity will have FQ enabled and more than E1 will be FIFO.


Hope to help,

Correct Answer
Giuseppe Larosa Fri, 07/23/2010 - 03:42
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Sarah,


I think the SW queues are created and used before congestion happens otherwise how we could provide better QOS to some traffic classes?



My opinion is based on many tests I did of CBWFQ on C7500 and MDRR schedulers on 12000 routers using traffic generator instruments like SMARTBITS or Agilent router tester. The first instrument has the limit to be not able to change the traffic load during the test, but Agilent router tester can do this.


This is my opinion and this can be true only of these high end routers, however a show policy-map interface shows packets for each traffic class on an interface where CBWFQ is applied outbound also if the link is not congested.


In real networks congestion happens without any advice what about the packets already in the FIFO HW queue ?


I agree that the HW queue is still there but minimized in size (expressed as a number of packets) and that may be a sort of threshold mechanism might be used to bypass the SW queues when average load is low. But I don't know how this could handle the sudden change to congestion state, when the thr threshold is reached we start to send packets to SW queues instead of sending them directly to the HW queue.



Hope to help

Giuseppe

sarahr202 Fri, 07/23/2010 - 19:01
User Badges:
  • Bronze, 100 points or more

Hi Giuseppe

How are you doing?


Qos should kicks in when router/node experiences a congestion.  So if there is no congestion, there is no  need to prioritize the packets in order to send them out of interface.  I am still trying to understand the responses i received.

Please correct me if  i am wrong.


Based on your response, i understand when you create qos configuration on router/node, you also create software queues. whether those will be used depends upon if router/node experiences congestion on outgoing interface.  Am i correct?


What queues  can be created when qos is used?  Are  all queues created as result of qos, software queues or they can be hardware queues?

what queues are hardware queues?   I  found out some such as fifo etc.





Thanks and have a nice weekend.

sarahr202 Sat, 07/24/2010 - 08:50
User Badges:
  • Bronze, 100 points or more




Hi Giuseppe.


http:// wiki.nil.com/File: Queuing_Software_Queues.jpg


Please look  at above link.( Sorry , i tried to paste the link but couldn't do it, must be some glitch on this computer)




I quote from the above link.


When to queue:


" The interface only interrupts the cpu when it needs more packet to transmit"



Implementation details:


"  When the interface transmit a packet,it also polls the next descriptor to figure out if there is another packet to be transmitted.


if the next tx descriptor indicates a packet is available,the transmission process continues with the next packet,otherwise the transmission circuitry enters the idle state.

In the idle state,the interface hardware periodically polls the next descriptor and starts the transmission as soon as it finds a descriptor marked as belonging to the interface."


===============================



I am confused here. What interface hardware does when it does not find any packet in tx ring, will it interrupt the cpu asking for more packets as mentioned under " When to queue" in this post or will it poll the periodically as mentioned under " implementation detail"



the above link does explains how the software queues are used even  when the link is not congested.  That might be the reason  you found software queue being used though there is no congestion on the link.



Thanks and have a good weekend.

Correct Answer
Giuseppe Larosa Sat, 07/24/2010 - 13:24
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Sarah,

you have found a good link by a well known author ( Pepelnjak is also author of books and writes in many places including his own blog iosinternals and on nanog forums)


>> The hardware queue shared between the CPU and the interface chipset cannot be reordered, as the CPU has no control over the operation of the interface hardware that might pick up a packet at any time. Once a packet enters the hardware queue, its position in the queue cannot be changed.


This is a clear description of my doubts: once the packet is in the HW queue is too late to provide any QoS differentiation.


For this reason the HW queue should get packets from the output of the SW queues also when congestion is not present it provides a way to give priority to small VoIP packets for example.


The HW queue size is minimized to only a few packets when SW Queues are enabled, this is reported also on some of those books that say that when the link is not full the SW queues are bypassed.


but if the HW queue is 4 packets "congestion" = 4 packets in the HQ queue or tx ring as it is called!


About your questions:

it is difficult to describe router internals:

packets are placed in a shared memory waiting to be transmitted the interface HW takes them from this memory and move them to the tx ring.

Actually in modern routers interrupts to main cpu are minimized by using the mechanism described above ( a packet memory)

The mentioned descriptors are internal tags added to the packet to allow low level and efficient moving of packets from packet memory to interfaces.


To be more practical: in my tests I usually had many traffic flows belonging to different traffic classes. To demonstrate good behavior of schedulers like CBWFQ  or MDDR the offered traffic had a total volume slightly more then outgoing interface speed (101% to 103%) but only one traffic class was over its limit. The expected result is that only packets of that exceeding class are dropped.

However, even using a load variable in time with the appropriate instruments results are in line with the idea that SW queues are not bypassed.



Have a nice weekend you too.


I' m more busy in this period so I'm less present in the forums




Hope to help

Giuseppe

Correct Answer
amahendi Fri, 07/23/2010 - 07:39
User Badges:
  • Cisco Employee,

" Will  received packet still get placed in some queue? or to put simply is there any queue associated with outgoing interface on router when no qos is configured on a router?"


Yes, there is always a queue associated with every L3 interface on a router. This is the interface hold-queue in the outbound direction. For interfaces <= ~ 2 Mbps, the queueing defaults to fair-queue; over 2 Mbps, it's fifo.


The interface hold-queue, like the queues associated with a QoS policy, is a software queue.


As for the packet flow:


- CEF switched traffic: During the tx_interrupt, packets are placed on the tx_ring or the so-called 'hardware queue'. This is basically the buffers present on the PLIM. The tx_ring is a FIFO structure and it's size various according to HW / media.


- Process switched traffic: Process switched traffic, like traffic originated form the router itself, *always* go through the 'software' queues(irrespective of whether there is congestion or not), before being placed on the tx_ring.


- An interface getting 'congested' basically means, the tx_ring is overrun. When this happens, there is backpressure from the tx_ring so that CEF switched packets are queued in the software queued (intf hold-queue or QoS queues). Hence, any software queueing is invoked by this 'backpressure' mechanism. This mechanism itself varies depending on the hardware, it can either be the tx_ring getting overrun (this is the case for most lower-end platforms) or can be an internal shaper / ASIC that provides this mechanism (like in the case of SIP / SPA).


- Abhi

Actions

This Discussion