QoS - CBWFQ & DSCP

Unanswered Question
Jul 12th, 2008

Hi,


I've been trying to figure this out for a while and I keep coming up against different answers:


Do packets matched by a single CBWFQ, which however have different DSCP markings, get prioritised out of the queue based on their DSCP markings or does the queue become a FIFO queue?


If you configure WRED on the CBWFQ it will drop packets based on their DSCP markings, so it would make sense that the packets get priority out of the queue based on their markings.


Is it dependent on the hardware? (eg. does it have to be a 7600 to not become FIFO?)


The other question I had was to do with the "bandwidth" command, does the "bandwidth" command have to configured on a CBWFQ to give it a higher priority than another CBWFQ, even if the second CBWFQ is matching on a lower priority DSCP value?


I thought I knew the answers but now I'm not so sure.


Any help would be much appreciated.


Regards,

Kaveh

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
dhananjoy chowdhury Sat, 07/12/2008 - 23:50

Yes indeed it does match and transmit packets on the basis of priority set as per the configuration based on DSCP markings and it doesnot have to do anything with the hardware.


using the bandwidth command lets u specify how much of the b/w buffer one would like to assign to a particular group. Though at anytime, a higher priority packet would be put first in queue for transmission as compared to a lower priority packet when processed at same time. However, if a packet is already in queue for transmission, it will not be surpassed by a higher priority packet

kaveh.azadi Sun, 07/13/2008 - 00:28

Hi Dhananjoy,


Thank you for your reply. This was also my understanding, hence the term CBWFQ. Unfortunately a response I received from the Cisco TAC on a similar issue previously and comments made in the CCIE R&S exam guide (3rd edition - page 441 for anyone interested) made me question it.

Have you by chance come across any documents that clearly state this? Everything I've found so far is either vague or doesn't go in to that depth, which is rather frustrating as this is key concept in any large QoS deployment.


Regards,

Kaveh

n.nandrekar Sun, 07/13/2008 - 00:50

hi Kaveh!

For different behaviors for different DSCP markings, it has to be configured to do so. If a single class matched a set of DSCP values, they all will be assigned the same priority and indeed use FIFO inside that same class. There is no priority assigned to the packets within the same class (except for class-default where WFQ is applied on per-flow basis). The difference between WFQ and CBWFQ is that WFQ applies on Per-flow basis where as CBWFQ applies per-class basis. All packets in the same class are assigned the same weight depending on the "bandwidth" and "queue-limit" configured for that class.


http://www.cisco.com/en/US/docs/ios/12_0t/12_0t5/feature/guide/cbwfq.html


http://www.cisco.com/en/US/tech/tk39/tk824/technologies_tech_note09186a0080093d62.shtml


the above links give more details on this. following is an extract from the same links :

" For CBWFQ, which extends the standard WFQ fair queueing, the weight specified for the class becomes the weight of each packet that meets the match criteria of the class. Packets that arrive at the output interface are classified according to the match criteria filters you define, then each one is assigned the appropriate weight. The weight for a packet belonging to a specific class is derived from the bandwidth you assigned to the class when you configured it; in this sense the weight for a class is user-configurable.


After the weight for a packet is assigned, the packet is enqueued in the appropriate class queue. CBWFQ uses the weights assigned to the queued packets to ensure that the class queue is serviced fairly. "


Also "CBWFQ allows you to specify the exact amount of bandwidth to be allocated for a specific class of traffic. Taking into account available bandwidth on the interface, you can configure up to 64 classes and control distribution among them, which is not the case with flow-based WFQ.


Flow-based WFQ applies weights to traffic to classify it into conversations and determine how much bandwidth each conversation is allowed relative to other conversations. for flow-based WFQ, these weights, and traffic classification, are dependent on and limited to the seven IP Precedence levels. "


For the "Bandwidth" command >>>>>> this command has nothing to do with giving Priority to the traffic. What it does is reserves the minimum amount of bandwidth available of traffic. So, in cases of congestion, the bandwidth specified in the command will be available for that class. If this is unused, it will be given to other classes.

So the packet scheduler will be configured so that the class gets that amount of bandwidth every time instant.


>>> The "priority" command is required to give priority to a particular traffic-class. It ensures that every time instance the first slot is reserved for the priority traffic.


Hope this makes things clear.


Regards,

niranjan

(Please rate helpful posts)

n.nandrekar Sun, 07/13/2008 - 01:02

sorry! I guess my post overlapped with yours.

The term CBWFQ comes from the fact that it overcomes the scalability limitations of WFQ by applying weights to packets on a per-class basis than per-flow basis. SO it is still WFQ but just that it distinguishes based on classes.

Whence Class Based WFQ.


Also regarding your original post, its always FIFO inside the class based queue. So thats independent of the platform. There are platform / software limitations on who supports CBWFQ though....


Also you can apply WRED inside a CBWFQ as a tail drop policy so that you can use different DSCP values in the same class for different tail drops. this could in one way achieve higher priority for some DSCPs in the same class by dropping other DSCPs more aggrissively but whichever pass through as still FIFO.


Regards,

Niranjan

(pls rate helpful posts)

Joseph W. Doherty Sun, 07/13/2008 - 03:52

"Also regarding your original post, its always FIFO inside the class based queue."


NB: except when fair-queue is configured within the class.

Joseph W. Doherty Sun, 07/13/2008 - 04:27

"All packets in the same class are assigned the same weight depending on the "bandwidth" and "queue-limit" configured for that class. "


Didn't think queue-limit has influence with weight assignment. Could you provide a documentation reference?

n.nandrekar Sun, 07/13/2008 - 04:44

Hi Joseph!

I didnot refer to any specific documentation for that. Since changing the "bandwidth" automatically changes the default queue-limit for the class, I thought that they both would affect the weight.

But after you pointing this out, I referred the available documentation on cisco, and :

"CBWFQ allows you to define traffic classes and apply parameters, such as bandwidth and queue-limits, to these classes. The bandwidth you assign to a class is used to calculate the "weight" of that class."


So probably I am wrong on that... But just as we are on this point, could you please explain how the queue-limit and bandwidth are interlinked? Since changing the bandwidth on the class changes the queue-limit on the queue, shouldnt that affect?

Or is it that since a queue is always in terms of "no. of packets" as its the address references to the packets thats actually in the queue, that it assumes a "default size of the packets" to calculate the queue-length.

More specifically how do queue-limit and bandwidth inter-relate? I have a certain picture in my mind and it would be great if you could shed more light on it.


Regards,

Niranjan

Joseph W. Doherty Sun, 07/13/2008 - 05:11

just as we are on this point, could you please explain how the queue-limit and bandwidth are interlinked? Since changing the bandwidth on the class changes the queue-limit on the queue, shouldnt that affect?

Or is it that since a queue is always in terms of "no. of packets" as its the address references to the packets thats actually in the queue, that it assumes a "default size of the packets" to calculate the queue-length.

More specifically how do queue-limit and bandwidth inter-relate? I have a certain picture in my mind and it would be great if you could shed more light on it.


Some platforms, many of the smaller routers, appear to always use a default queue-length (often 64) regardless of bandwidth configuration. Those that don't, seem to adjust queue-length based on some calculation of bandwidth and the buffer resouces of the platform. Probably only Cisco could explain the calculation. Have seen it define FIFO queues very small, e.g. 2 packets, and very large, e.g. in the thousands. On such platforms, I define queue-length, either based on BDP or my intended drop policy.


Don't believe I've seen packet's class weight change at all based on queue-length. (Which would agree with the documention you quoted.)


Does this help with your mental picture?

Joseph W. Doherty Sun, 07/13/2008 - 04:56

"Do packets matched by a single CBWFQ, which however have different DSCP markings, get prioritised out of the queue based on their DSCP markings or does the queue become a FIFO queue?"


If the CBWFQ class queue is a FIFO queue, then TOS markings have no influence on that class's packet ordering. However, class queues are not always FIFO queues. When they're not, TOS markings might change that class's packet ordering.


"If you configure WRED on the CBWFQ it will drop packets based on their DSCP markings, so it would make sense that the packets get priority out of the queue based on their markings. "


If you consider RFC usage of DSCP bases priority on class (or the former IP Precedence bits), e.g. AF2x vs. AF4x, but drop precedence on drop priority, e.g. AFx1 vs. AFx3, one could argue it doesn't always makes sense.


"Is it dependent on the hardware? (eg. does it have to be a 7600 to not become FIFO?)"


Platform and software do often have much to do with what can be configured. For instance, on the 6500/7600 certain WAN cards support CBWFQ class fair-queue within "normal" classes where most platforms only support it within the class-default. (The 7500 also support FQ within non-class-default classes.)


"The other question I had was to do with the "bandwidth" command, does the "bandwidth" command have to configured on a CBWFQ to give it a higher priority than another CBWFQ, even if the second CBWFQ is matching on a lower priority DSCP value?"


Assuming "bandwidth" isn't required for a FIFO CBWFQ class, I'm unsure what weight would be given to the class's packets. I would suspect a FIFO class's packets would all have the same weight regardless of the TOS markings, but would highly recommend against building a policy working based on a particular platform's default workings since such often vary between platform and version. Also consider you might be making the assumption, or relying on the platform to assume, AF4x is better than AF1x, but really AF4x is different from AF1x. (Unlike IP Prececence 4 is "better" than IP Precedence 1.) Your QoS policy determines how they differ. CBWFQ allows you do implement the policy.

kaveh.azadi Sun, 07/13/2008 - 06:31

Hi gents,


Firstly thank you for your informative feedback and efforts in answering the questions.


So based on the discussions and all the documentation, the answer to the CBWFQ and DSCP interaction is that on most platforms (besides the 7500/7600, which can be configured with flow-based WFQ on all queues) regardless of the IPP/DSCP markings the scheduling inside a single CBWFQ is FIFO. However, the class-default when configured with the "fair-queue" command becomes a flow-based WFQ, which does take into account IPP markings.

WRED can then be configured on either the CBWFQ or class-default to take into account IPP or DSCP priorities when deciding which packets to drop if software queue is becoming congested.


With regards to the "bandwidth" command, it doesn't have to be configured on a CBWFQ class in order for that class to be allocated a larger portion of the available bandwidth dedicated to it by the scheduler during congestion, as long as that class's DSCP/IPP markings are higher than other CBWFQ classes that are competing for the same bandwidth.


The reason for these questions is that to deploy a QoS policy across a very large any-to-any WAN, I believe it's impractical to try and develop a one-to-one DSCP to Class mapping. This is because the overhead of maintaining and tweaking the appropriate “bandwidth” for each class at each site would be rather tedious and very time consuming. Instead, I think it would be a better idea to group all critical applications based on their DSCP markings into a single class and reserve that class a large portion of the bandwidth across the WAN link. WRED could then be used on any non-critical class (such as class-default) to ensure higher priority classes get maximum throughput.


Thanks again for your time.


Regards,

Kaveh

Joseph W. Doherty Sun, 07/13/2008 - 12:07

"With regards to the "bandwidth" command, it doesn't have to be configured on a CBWFQ class in order for that class to be allocated a larger portion of the available bandwidth dedicated to it by the scheduler during congestion, as long as that class's DSCP/IPP markings are higher than other CBWFQ classes that are competing for the same bandwidth."


Unsure that's true.


"The reason for these questions is that to deploy a QoS policy across a very large any-to-any WAN, I believe it's impractical to try and develop a one-to-one DSCP to Class mapping. This is because the overhead of maintaining and tweaking the appropriate “bandwidth” for each class at each site would be rather tedious and very time consuming. Instead, I think it would be a better idea to group all critical applications based on their DSCP markings into a single class and reserve that class a large portion of the bandwidth across the WAN link. WRED could then be used on any non-critical class (such as class-default) to ensure higher priority classes get maximum throughput."


If you're defining bandwidth with absolute numbers, you're correct, very tedious to maintain. However, if you base priority on relative ratios and if the platform supports percentaged based bandwidths, not much effort supporting one policy on all interfaces, all platforms, especially if only one class per DSCP class.


One issue relying on WFQ weights, percentage of bandwidth per flow, even without different weights, varies based on number of flows.


I would suggest starting with a policy that just places everything into a class-default fair-queue. If there's real-time traffic, drop it into a LLQ class. If there's a mix of bulk traffic and interactive traffic, try to place the bulk traffic into it's own FIFO class with minimum bandwidth gurantee.


e.g. (an example of a simple/complex policy)

(NB: syntax might be incorrect)


class-map match-any real-time

match ip dscp EF

(etc.)


class-map match-any bulk-traffic

match protocol FTP

(etc.)


class-map match-any scavenger

match protocol BitTorrent

(etc.)


policy-map general_CBWFQ


class real-time

priority percent 30


class bulk-traffic

bandwidth percent 10


class scavenger

bandwidth percent 1

police percent 5


class class-default

fair-queue


With regard to WRED, very difficult to use correctly. If bulk traffic is TCP traffic, RED might be used in an attempt to improve goodput.

kaveh.azadi Mon, 07/14/2008 - 07:50

Hi Joseph,


Thanks again for taking the time to reply. The QoS policy that I'm working on at the moment is rather large and detailed as it involves the use of a three tiered service policy to mark numerous applications, then to shape and specify queuing method and bandwidth according to the destination site's link speed and the type of application, and finally to map to the carrier's IPP classes. So the approach I've had to take is to use all the various QoS tools to hopefully achieve a satisfactory result without creating too much of a burden for our support teams.

We'll also be using Cisco's QPM 4.0 in order to bring some form of manageability to the process. What I'm hoping for is to be able to use QPM's reporting capabilities to monitor the policies and to tweak and adjust them if and when necessary.

All going as planned, I should also be able to verify all the items we've discussed in this posting as well.

I'll update the post when I know more.

Thanks for your help.


Regards,

Kaveh

Actions

This Discussion