cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2584
Views
14
Helpful
14
Replies

QoS on a 6509

lamav
Level 8
Level 8

Hi, folks:

This is the QoS configuration that is applied to some of the server interfaces on an access layer 6509 running 12(2)SXF18.

interface GigabitEthernet8/3
description Bowers VMWARE 1
switchport
switchport access vlan 752
switchport mode access
no ip address
wrr-queue cos-map 1 1 1
wrr-queue cos-map 2 1 0
wrr-queue cos-map 2 2 3
wrr-queue cos-map 3 1 4
wrr-queue cos-map 3 2 2
wrr-queue cos-map 3 4 6
wrr-queue cos-map 3 5 7

A server is plugged into this port.

The scheduling capabilities of this interface are the following:

  QOS scheduling:        rx-(1q8t), tx-(1p3q8t)

I'm a bit confused. There is 1 priority transmit and 3 regular transmit queues, and only one regular input queue, therefore, one can assume that what has been configured on this interface is output queuing/scheduling. Is that correct?

Isnt it peculiar to configure output scheduling on an interface that connects to a server/end-user?

Shouldnt output queuing/scheduling be configured on the uplinks to the distribution layer?

Lastly, how does one differentiate between the input queues and the output quesues when configuring scheduling? For example, given the line  wrr-queue cos-map 1 1 1, how does one know whether queue 1 in that this command is referencing queue1 of the outo\put queue or the sole input queue?

Thanks

Victor

14 Replies 14

Jon Marshall
Hall of Fame
Hall of Fame

lamav wrote:

Hi, folks:

This is the QoS configuration that is applied to some of the server interfaces on an access layer 6509 running 12(2)SXF18.

interface GigabitEthernet8/3
description Bowers VMWARE 1
switchport
switchport access vlan 752
switchport mode access
no ip address
wrr-queue cos-map 1 1 1
wrr-queue cos-map 2 1 0
wrr-queue cos-map 2 2 3
wrr-queue cos-map 3 1 4
wrr-queue cos-map 3 2 2
wrr-queue cos-map 3 4 6
wrr-queue cos-map 3 5 7

A server is plugged into this port.

The scheduling capabilities of this interface are the following:

  QOS scheduling:        rx-(1q8t), tx-(1p3q8t)

I'm a bit confused. There is 1 priority transmit and 3 regular transmit queues, and only one regular input queue, therefore, one can assume that what has been configured on this interface is output queuing/scheduling. Is that correct?

Isnt it peculiar to configure output scheduling on an interface that connects to a server/end-user?

Shouldnt output queuing/scheduling be configured on the uplinks to the distribution layer?

Lastly, how does one differentiate between the input queues and the output quesues when configuring scheduling? For example, given the line  wrr-queue cos-map 1 1 1, how does one know whether queue 1 in that this command is referencing queue1 of the outo\put queue or the sole input queue?

Thanks

Victor

Victor

Yes the wrr-queue commands are for the transmit queues as WRR is a queueing method used for transmitting the packets. The equivalent commands to the ones you have above for ingress would be "rcv-queue cos-map ..." although obviously as you point out the ingress does not have the same queueing structure -

6500 ingress queueing command reference

As for it being configured on a server port, well i don't know the logic of why that has been done. Perhaps it's just a default config or perhaps traffic is actually being prioritsed to the server.

Jon

Jon:

I'm glad you did that sanity check for me. Nice work on the rcv queue command. Now it makes sense.

You know, I've seen many such examples as the one I posted on here, and I always just accepted the configuration. I made an effort to understand what each command line is doing, but it never occurred to me why it is typical to configure output queuing on an interface that has a VoIP phone or server connected to it. I guess it is easy enough to understand that even an end-user port will have output queuing configured to ensure that no delay occurs, even at the very last hop. Although, I do think that with ethernet speeds, output queuing is less imperative as it would be on a lower bandwidth router interface.

As I said, the configuration commands make sense - nothing too mysterious, but parenthetically, it does suck that Cisco's CLI for implementing LAN QoS varies so much from platform to platform and from IOS version to IOS version. The commands differ as well as the approaches.

What really puzzles me, though, about the config I am looking at is that none of the ports configured for output scheduling have any bandwidth allocations or priority queing configured. The configuration I posted does nothing but place packets with certain CoS values in certain queues -- and thats it. The Queues do not have any scheduling/forwarding parameters defined, such as bandwidth, nor is priority queing configured.In fact, all the output queues are still set for the default, fifo.

So, what is the value of the QoS configuration? Am I missing something? I dont have to post the config. Suffice it to say that all the uplinks have the exact same CoS-to-queue mapping, with no bandwidth allocations or priority queuing configured

On the ingress side -- whether ingress from another network element or an end-user/server port -- the mls qos trust dscp commands are in place where applicable, so that part makes sense.

Hi Iamav,

What is the output of the "Show queueing interface x/x" command? Does that idenitfy Q4 as the priority queue? I had some similar confusion with a 6509, it was also set to "priority-queue cos-map 1 5" - but as this is a default setting, it does not show in the running config.

Cheers

Jonathan

J:

Upon executing the sh queing interface x/x command, I see that queue 4 is the priority queue.So, the administrator also did not take advantage of the priority queue.

Moreover, the bandwidth settings for the QoS-enabled interfaces are configured for each queue by the switch as defaults. So, you're right, it didnt show up in the running config because the administrator did not change the defaults and the defaults do not appear in the running config.

The good news is that I was conceptually correct: the enqueing plan for different CoS values alone does nothing without the corresponding bandwidth weights for each queue. It should have occurred to me, though, that the switch must configure default settings if the administartor doesnt customize, otherwise the technology wont work...

As far as scheduling, WRR will service each queue in a round robing fashion, but the weight (bandwidth privileges) assigned to each queue will dictate how many packets aer sent each time around. The priority queue, however, will always get serviced first if there is a packet in it. The difference between WRR with priority queuing and Priority Queing is that the priority queue does get policed so that it does not starve the lower priority queues.

OK, now I'm rambling...

Victor

lamav wrote:

Jon:

I'm glad you did that sanity check for me. Nice work on the rcv queue command. Now it makes sense.

You know, I've seen many such examples as the one I posted on here, and I always just accepted the configuration. I made an effort to understand what each command line is doing, but it never occurred to me why it is typical to configure output queuing on an interface that has a VoIP phone or server connected to it. I guess it is easy enough to understand that even an end-user port will have output queuing configured to ensure that no delay occurs, even at the very last hop. Although, I do think that with ethernet speeds, output queuing is less imperative as it would be on a lower bandwidth router interface.

As I said, the configuration commands make sense - nothing too mysterious, but parenthetically, it does suck that Cisco's CLI for implementing LAN QoS varies so much from platform to platform and from IOS version to IOS version. The commands differ as well as the approaches.

What really puzzles me, though, about the config I am looking at is that none of the ports configured for output scheduling have any bandwidth allocations or priority queing configured. The configuration I posted does nothing but place packets with certain CoS values in certain queues -- and thats it. The Queues do not have any scheduling/forwarding parameters defined, such as bandwidth, nor is priority queing configured.In fact, all the output queues are still set for the default, fifo.

So, what is the value of the QoS configuration? Am I missing something? I dont have to post the config. Suffice it to say that all the uplinks have the exact same CoS-to-queue mapping, with no bandwidth allocations or priority queuing configured

On the ingress side -- whether ingress from another network element or an end-user/server port -- the mls qos trust dscp commands are in place where applicable, so that part makes sense.

Victor

Interesting point about there being no bandwidth allocation etc. In this case the switch will use the defaults -

6500 transmit queue defaults

so as long as there are CoS values in the packets there will be default scheduling happening. Also worth checking interface as Jonathan suggested.

I completely agree about catalyst QOS being a pain but the problem is that QOS is done in hardware on switches and as each platform has differing capabilities so the QOS commands are different.

Jon

Jon:

Thanks and read my post above. :-) I have seen the glory! lol

Thanks gentlemen.

By the way, here is what seems like a great QoS document specifically for the 6500 and the 7600. Save it.

http://labs.spritelink.net/labs/documents/QoS/Cisco_QoS_6500_7600_indepth_design.pdf

By the way, one other peculiar thing. The queing on the interface that has QoS enabled on it (WRR scheduling) still shows fifo. Why is that?

NYTAR777-SDS04#sh run int gi1/1
Building configuration...

Current configuration : 358 bytes
!
interface GigabitEthernet1/2
description
switchport
switchport access vlan 300
switchport mode access
no ip address
speed 100
duplex full
wrr-queue cos-map 1 1 1
wrr-queue cos-map 2 1 0
wrr-queue cos-map 2 2 3
wrr-queue cos-map 3 1 4
wrr-queue cos-map 3 2 2
wrr-queue cos-map 3 4 6
wrr-queue cos-map 3 5 7
end

NYTAR777-SDS04#
NYTAR777-SDS04#sh int gi1/2
GigabitEthernet1/1 is up, line protocol is up(notconnect)
  Hardware is C6k 1000Mb 802.3, address is 0018.b916.9910 (bia 0018.b916.9910)
  Description: TT7513 TO DALLAS FE 0/0
  MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s
  input flow-control is off, output flow-control is off
  Clock mode is auto
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 36w6d, output 36w6d, output hang never
  Last clearing of "show interface" counters 1d01h
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts (0 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     0 packets output, 0 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out
NYTAR777-SDS04#

I wish I knew...Im getting the same result on my lab switch...

Victor

Can you post output of "sh queueing int gi1/2"

Jon

Hi, Jon:

I should hsave mentioned that the output of that command shows the correct queueing strategy.

#sh queueing int gi 1/2
Interface GigabitEthernet1/2 queueing strategy:  Weighted Round-Robin

I just dont know why its not reflected in the sh int output. I also cant seem to find any information in the literature Ive been reading that would explain it either...

The sh int output queueing reference may be elaborating on the queueing stragetgy within a single queue and not the queueing strategy between queues, which, parenthetically, for the class-default queue in CBWFQ can be changed from fifo to WFQ, but that doesnt seem likely...

I dont want to perseverate over minutiae, but this behavior does seem counter-intuitive.

Thanks

Victor

lamav wrote:

Hi, Jon:

I should hsave mentioned that the output of that command shows the correct queueing strategy.

#sh queueing int gi 1/2
Interface GigabitEthernet1/2 queueing strategy:  Weighted Round-Robin

I just dont know why its not reflected in the sh int output. I also cant seem to find any information in the literature Ive been reading that would explain it either...

The sh int output queueing reference may be elaborating on the queueing stragetgy within a single queue and not the queueing strategy between queues, which, parenthetically, for the class-default queue in CBWFQ can be changed from fifo to WFQ, but that doesnt seem likely...

I dont want to perseverate over minutiae, but this behavior does seem counter-intuitive.

Thanks

Victor

Victor

I dont want to perseverate over minutiae, but this behavior does seem counter-intuitive.

Well quite often it's this attention to detail that helps us understand things properly so i think it's a valid question. I don't have a 6500 to hand at the moment but i suspect you are right in what you say. I think the "sh int" reports fifo because per queue that's exactly what it uses regardless of how you have split the bandwidth etc.. between queues.

Jon

Hey, Jon:

After exhausting all avenues, I opened a TAC case and I was told that the queueing strategy indication under the "sh interface" command output refers to the queueing strategy for traffic heading to the RP - at least in the 6500. That doesnt sound right at all, but I didnt want to press the issue.

I think the answer is that the default software queueing on all ethernet interfaces is fifo and this is what is being referenced when we execute the "sh int" command. So, unless we change the software queueing strategy by configuring specific QoS scheduling  (PQ, CQ, WFQ, CBWFQ, LLC), the software queueing indicator will remain the same.

On the other hand, when we execute the "sh queuing int x/x" command, we are specifically querying the switch with regard to the hardware queues.

By the way, even when we configure software queueing, there is still a single hardware fifo buffer after the software queues that stores the packets as they are being placed on the wire (serialization). This is not configurable. They call it the Transmit Queue, or TX Ring or just Hardware Queue. If there is no congestion, and therefore no need for the network device to implement congestion management, the packets bypass the software queues and are placed directly in this single hardware queue, where they are then serialized.

How this single queue relates to the hardware queues we configure with LAN QoS is something I am not sure about.

I think I'll put the issue to rest, buddy, unless you have a few more nuggets to lay on me. :-)

Thanks

Victor

[EDIT] On second thought, maybe the TAC tech was referring to queueing in software when he referred to traffic heading to the RP. [EDIT]

Hello Victor,

>> If there is no congestion, and therefore no need for the network device to implement congestion management, the packets bypass the software queues and are placed directly in this single hardware queue, where they are then serialized.

Some notes here:

this description is good for SW based routers and it might be not accurate for HW based multilayer switch like C6500.

Also for SW based routers I don't trust the sentence that SW queues are bypassed if there is no congestion.

If we configure a CBWFQ on a router like C7200 or C7500 or a GSR 12000 ISE linecard that supports MQC we see that sh policy-map int provides counters accurate for all classes even when there is no congestion.

Also congestion does not advertise itself in any  way: it arrives suddenly with traffic peaks how could we protect VoIP or other traffic if the router would need the time to detect congestion and to start to use the SW configured queues?

I made years ago extensive tests on DWFQ first and then on CBWFQ and MDRR (on GSR) using  traffic generators like SMARTBITS or router tester Agilent (where I could dynamically change the load level on the fly) with multiple classes and I can say that until traffic overload is not very heavy ( less then 130% of WAN interface just to say with non TCP traffic) the scheduler respects the configured BW values among classes and that what is left free by one class can be used by other classes.

Clearly, the output of sh int on C6500 regards queueing is misleading, packets that need to be sent to RP can be among the received packets on the interface, among them there ARP packets, routing protocol packets and all packets that need to be punted to RP.

Hope  to help

Giuseppe

Hi, Giuseppe:

Long time. Hope all is well.

Im a bit confused about what you agree with because you first say that my explanation may be true for SW-based routers and then you say you dont trust it.

Regardless, congestion within the context of queueing refers to whether there is availability in the Hardware (TX Ring) Queue.

Without QoS scheduling being enabled on an interface, the hardware queue length is longer. The longer the queue, the greater the serialization delay.

When QoS scheduling is configured and a service policy is applied to that same interface, the Hardware Queue length is automatically shortened to accommodate the queueing delay introduced by the QoS scheduler itself. Packets face delay as they are enqueued in the software queues. This is why the Hardware Queue is leveraged before the software queues.Take note that the scheduling mechanism delineated by the service policy applies only to the forwarding of packets enqueued in software queues. The service policy is transparent to the hardware Queue.

On the other hand, it is true that queueing on the TX Ring introduces a serialization delay that is directly proportional to the depth of the ring. An excessive serialization delay can impact latency budgets for delay-sensitive applications such as voice.

So, the size of the transmit ring needs to be small enough to avoid introducing latency due to software queueing, and it needs to be large enough to avoid drops and a resulting impact to TCP-based flows.

All that having been said, the default Hardware Queue setting when a service policy has been applied on most router platforms is 2 packets. So, for example, given an interface with a clockrate of a T1 link, if 2 packets with a maximum byte size of 1500 bytes were to occupy the Hardware Queue, the serialization delay would only be 15 ms. The priority queueing mechanism of the applied software scheduler would then send the VoIP packet next.

Given that the majority of end-to-end delay can be attributed to serialization and software queueing delays of each router hop (leaving switching delays aside for a moment), and given that the industry standard for maximum acceptable one-way delay for a VoIP packet is 150 ms, it would take about 10 hops over 10 serial links to reach that limit. That is a pretty unlikely scenario to boot. Moreover, Cisco recommends 200 ms of maximum one way latency for VoIP packets, so that makes the Hardware queueing delays even more negligible. This is why a delay-intolerant packet, like a VoIP packet, can still enjoy minimal queueing delays and latency, even when the Hardware Queue is utilized first.

Victor

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: