QoS on Nexus 7000...

Unanswered Question
May 21st, 2010

Does anyone have a tried and true QoS configuration for a Nexus 7k?

I'm trying to replicate the standard voice/priority, signalling/bandwidth, default WRED type policy.

Here is what I hacked together...

class-map type qos match-any 1p7q4t-out-q2
  match dscp 24,26
class-map type qos match-all 1p7q4t-out-pq1
  match dscp 46
policy-map type queuing 10G_Metro_OUT
  class type queuing 1p7q4t-out-pq1
    priority level 1
  class type queuing 1p7q4t-out-q2
    bandwidth remaining percent 3
  class type queuing 1p7q4t-out-q-default
    random-detect cos-based

Am I missing anything?  Is there a way to police the priority queue?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 0 (0 ratings)
ganeshh.iyer Fri, 05/21/2010 - 11:02

Does anyone have a tried and true QoS configuration for a Nexus 7k?

I'm trying to replicate the standard voice/priority, signalling/bandwidth, default WRED type policy.

Here is what I hacked together...

class-map type qos match-any 1p7q4t-out-q2
  match dscp 24,26
class-map type qos match-all 1p7q4t-out-pq1
  match dscp 46
policy-map type queuing 10G_Metro_OUT
  class type queuing 1p7q4t-out-pq1
    priority level 1
  class type queuing 1p7q4t-out-q2
    bandwidth remaining percent 3
  class type queuing 1p7q4t-out-q-default
    random-detect cos-based

Am I missing anything?  Is there a way to police the priority queue?

Hi,

Check out the below link for Modular QoS CLI (MQC) objects that can be used for configuring QoS features using the Cisco Nexus 7000 Series NX-OS software:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_1/nx-os/qos/configuration/guide/statistics.html

Hope to Help !!

Ganesh.H

gotney@consulta... Wed, 07/14/2010 - 12:26

I needed to implement a QoS architect to support a Call Manager/Cisco IPT environment.  We worked with a Cisco SE and tried to use the 6500 Voice Deployment Guide white paper as a base, (http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd804dc7ae.pdf) and converted it into Nexus 7K configs.

I can’t promise it’s correct, and would like some feedback from others.  The frustration for us field engineers is that there is plenty of reference material available for QoS on a 6500, 3750, or other switch, but not much exists yet for the Nexus line.  There is some decent theory stuff, but nothing that provides actual config script like you see in the white paper mentioned above.

Again, please review and post back with any changes/suggestions.


One gig card:

class-map type queuing match-any 1p3q4t-out-pq1
  match cos 5
class-map type queuing match-any 1p3q4t-out-q2
  match cos 2-4,6-7
class-map type queuing match-any 1p3q4t-out-q3
  match cos 0
class-map type queuing match-any 1p3q4t-out-q-default
  match cos 1
!
policy-map type queuing ONE_GIG-QUEUE-OUT
  class type queuing 1p3q4t-out-pq1
    priority level 1
    queue-limit percent 30
  class type queuing 1p3q4t-out-q2
    random-detect cos-based
    random-detect cos 7 minimum-threshold percent 90 maximum-threshold percent 100
    random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 90
    random-detect cos 3 minimum-threshold percent 70 maximum-threshold percent 80
    random-detect cos 2,4 minimum-threshold percent 60 maximum-threshold percent 70
    queue-limit percent 40
    bandwidth remaining percent 55
  class type queuing 1p3q4t-out-q3
    random-detect cos-based
    random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100
    queue-limit percent 25
    bandwidth remaining percent 40
  class type queuing 1p3q4t-out-q-default
    random-detect cos-based
    random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100
    queue-limit percent 5
    bandwidth remaining percent 5
!
interface eth 1/1
  service-policy type queuing output ONE_GIG-QUEUE-OUT


Ten gig card:

class-map type queuing match-any 1p7q4t-out-pq1
  match cos 5
class-map type queuing match-any 1p7q4t-out-q2
  match cos 7
class-map type queuing match-any 1p7q4t-out-q3
  match cos 6
class-map type queuing match-any 1p7q4t-out-q4
  match cos 3
class-map type queuing match-any 1p7q4t-out-q5
  match cos 2
class-map type queuing match-any 1p7q4t-out-q6
  match cos 4
class-map type queuing match-any 1p7q4t-out-q7
  match cos 0
class-map type queuing match-any 1p7q4t-out-q-default
  match cos 1
!
policy-map type queuing TEN_GIG-QUEUE-OUT
  class type queuing 1p7q4t-out-pq1
    priority level 1
    queue-limit percent 30
  class type queuing 1p7q4t-out-q2
    random-detect cos-based
    random-detect cos 7 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
  class type queuing 1p7q4t-out-q3
    random-detect cos-based
    random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
  class type queuing 1p7q4t-out-q4
    random-detect cos-based
    random-detect cos 3 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
  class type queuing 1p7q4t-out-q5
    random-detect cos-based
    random-detect cos 2 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
  class type queuing 1p7q4t-out-q6
    random-detect cos-based
    random-detect cos 4 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 20
    queue-limit percent 10
  class type queuing 1p7q4t-out-q7
    random-detect cos-based
    random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 25
    queue-limit percent 25
  class type queuing 1p7q4t-out-q-default
    random-detect cos-based
    random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100
    bandwidth remaining percent 5
    queue-limit percent 5
!
interface eth 10/1
  service-policy type queuing output TEN_GIG-QUEUE-OUT

bradburyh Wed, 07/21/2010 - 11:16

Did you ever figure out whether this solution worked?  I am assuming you put this queueing policy on all trunks to other switches?

  I am looking at a one way audio issue ever since my customer moved their gateway from their old CATOS 6500 to the Nexus switch (I

think they also upgraded the gateway at the same time).  And I am wondering if there is a possibility that QOS issues are occuring.

Thanks,

gotney@consulta... Wed, 07/21/2010 - 11:37

Unfortunately, the "voice" portion of the project was delayed for budgeting reasons, so the specific QoS settings below were never tested on the production network.  The voice portion was just re-approved and we should pick back up on it within a month or two, however, we'll be adding a couple TelePresense CTS-1000 and CTS-500 systems also which will mean updating the configs to allow for larger Priority Queues.  I'll post the resulting configs once we get things going.

One way audio?  I always start that troubleshooting by taking some sniffer traces at different points in the network to see if one voice stream is dropped or if the DSCP is stripped.  The Nexus trusts DSCP by default, so it *shouldn't* be stripping those QoS values there.  In my experience, as a "network guy” asymmetric routes through a WAN or Firewall issues (or both!) are good candidates to cause one way audio..

bradburyh Wed, 07/21/2010 - 11:42

I would normally have all the same ideas you suggested,  However the gateway sits in the same vlan as the phones and as the CallManager.  There is no firewall, etc in between.  And it is not all calls, so trying to think out of the box.  I am trying to get sniffer traces from both the port connected to the gateway and the phone. The hardest part is that I am not local to the site, so getting all the neccesary information has been difficult and replicating the issue has also been difficult.  I am also looking into DSP or other issues on the gateway.

aamercado Wed, 10/13/2010 - 11:27

10 Gig QoS seems to work fine...still testing:

1 Gig QoS has bug which I seem to have hit but is unreproducible in Cisco labs testing:

CSCtf76095    N7k QoS: ERRMSG: Total bandwidth remaining is greater than 100%

The below changes worked for me. My topology was a VPC to a 65XX and I got an error when I applied it to my Port-Channel. It didn't populate to my interface so I had to manually apply it to the each member of the port-channel.

interface port-channel27
  description ****S-EDGE-7-IDF-WEST-C6509****
  switchport
  switchport mode trunk
  vpc 27
  switchport trunk allowed vlan 71-72,171-172
  spanning-tree port type normal
  logging event port link-status
  logging event port trunk-status
  service-policy type queuing output ONE_GIG-QUEUE-OUT

class-map type queuing match-any 1p3q4t-out-pq1

  match cos 5

class-map type queuing match-any 1p3q4t-out-q2

  match cos 2-4,6-7

class-map type queuing match-any 1p3q4t-out-q3

  match cos 0

class-map type queuing match-any 1p3q4t-out-q-default

  match cos 1

!

policy-map type queuing ONE_GIG-QUEUE-OUT

  class type queuing 1p3q4t-out-pq1

    priority level 1

    queue-limit percent 30

  class type queuing 1p3q4t-out-q2

    random-detect cos-based

    random-detect cos 7 minimum-threshold percent 90 maximum-threshold percent 100

    random-detect cos 6 minimum-threshold percent 80 maximum-threshold percent 90

    random-detect cos 3 minimum-threshold percent 70 maximum-threshold percent 80

    random-detect cos 2,4 minimum-threshold percent 60 maximum-threshold percent 70

    queue-limit percent 40

    bandwidth remaining percent 50 (=½)

  class type queuing 1p3q4t-out-q3

    random-detect cos-based

    random-detect cos 0 minimum-threshold percent 80 maximum-threshold percent 100

    queue-limit percent 25

    bandwidth remaining percent 25 (=¼)

  class type queuing 1p3q4t-out-q-default

    random-detect cos-based

    random-detect cos 1 minimum-threshold percent 80 maximum-threshold percent 100

    queue-limit percent 5

    bandwidth remaining percent 25 (=¼)

Bandwidth Ratio Values:

1/2 = 50

1/3 = 33

1/4 = 25

1/5 = 20

Krzysztof.Zaleski_2 Fri, 08/26/2011 - 13:51

Hello,

Has anyone of you managed to configure QoS on N7K already? It's been a while since you posted this message, and I can see that there is still not many good documents/recommendations for QoS on N7K. My concern is how to properly use "queue-limit", "random-detect", and "bandwidth" commands in certain queues and scenarios. I have seen many examples where all commands are used, but why? From what I understand, "bandwidth" guarantees some minimum part of available bandwidtht during congestion, just like in plain CBWFQ. We can add random-detect for different COS values to proactively manage congestion. But, what is the real need to limit the queue with "queue-limit", except priority class?

In the above example there is a "queue-limit percent 40" which means packets are tail-dropped at that level of total queue depth (6,2MB per port for 1G port), but there is still "random-detect remaining percent 50", which means a) 50 percent of total available buffer size (excluding priority class, as it is remaining percent, but still it's more than queue-limit!!!), or b) 50 percent of that "queue-limit percent 40"?

Cheers,

Krzysztof

ingramchris Thu, 03/15/2012 - 18:08

I have this basic QoS policy I'm using for all of our 7k switches (2 at each data center that are basically connected by a UVN ring).

It has served the purpose until lately we need to classify some datadomain traffic and utilize another queue.

policy-map type queuing BANDWIDTH

  class type queuing 1p3q4t-out-pq1

    priority level 1

    queue-limit percent 30

  class type queuing 1p3q4t-out-q2

  class type queuing 1p3q4t-out-q3

  class type queuing 1p3q4t-out-q-default

    random-detect cos-based

    random-detect cos 0-1 minimum-threshold percent 60 maximum-threshold percent 70

    random-detect cos 2 minimum-threshold percent 80 maximum-threshold percent 90

    random-detect cos 3-4 minimum-threshold percent 90 maximum-threshold percent 100

    bandwidth remaining percent 70

VoIP goes to the priority queue (1p3q4t-out-pq1).

Everything else goes to the default queue (1p3q4t-out-q-default) and is subject to random dropping by  WRED.

Now we need to move our CoS 2 traffic to queue 2 and I have only one problem.  I cannot edit exisiting class-maps of type queuing and I cannot create new ones  (only type qos, or network-qos). 

Here's what the class-map confiuration looks like:

    class-map type queuing match-any 1p3q4t-out-pq1
      Description: Classifier for egress priority queue of type 1p3q4t
      match cos 5-7

    class-map type queuing match-any 1p3q4t-out-q2
      Description: Classifier for egress queue 2 of type 1p3q4t

    class-map type queuing match-any 1p3q4t-out-q3
      Description: Classifier for egress queue 3 of type 1p3q4t

    class-map type queuing match-any 1p3q4t-out-q-default
      Description: Classifier for egress default queue of type 1p3q4t
      match cos 0-4

I just need to figure out how to match CoS 2 under class-map type queuing match-any 1p3q4t-out-q2.

Does anyone know how?

Thanks,

Chris

ingramchris Thu, 03/15/2012 - 18:28

Now I have just re-written this policy to reflect my intentions of matching CoS 2 under the 2nd queue.

policy-map type queuing BANDWIDTH
class type queuing 1p3q4t-out-pq1
   priority level 1
   queue-limit percent 30
class type queuing 1p3q4t-out-q2
   bandwidth remaining percent 10
   random-detect cos-based
   random-detect cos 2 minimum-threshold percent 90 maximum-threshold percent 100
class type queuing 1p3q4t-out-q3
class type queuing 1p3q4t-out-q-default
   bandwidth remaining percent 60
   random-detect cos-based
   random-detect cos 0-1 minimum-threshold percent 60 maximum-threshold percent 70
   random-detect cos 3-4 minimum-threshold percent 90 maximum-threshold percent 100
ametcalf Thu, 06/21/2012 - 09:31

Chris

I guess you have answered your own questions by now, but I am not sure that issuing

policy-map type queuing BANDWIDTH

class type queuing 1p3q4t-out-q2

random-detect cos-based

random-detect cos 2 minimum-threshold percent 90 maximum-threshold percent 100

commands when no traffic of that cos value is mapped to the queue will help.

I assume you also from the default/admin VDC configured this

class-map type queuing match-any 1p3q4t-out-q2

  match cos 2

or did it actually work ?

For me I find a bit of a pain/restrcitive because this is then applied globally. The documentation suggests that you can copy the defaults and play with them but so far I have not managed, but I have not dedicated that muich time to it (just one night when I had to get it in and working very quickly)

Actions

Login or Register to take actions

This Discussion

Posted May 21, 2010 at 10:57 AM
Stats:
Replies:11 Avg. Rating:
Views:14063 Votes:0
Shares:0
Categories: Switches
+

Related Content

Discussions Leaderboard

Rank Username Points
1 15,007
2 8,150
3 7,725
4 7,083
5 6,742
Rank Username Points
165
82
69
65
55