Best Practices Metro Ethernet QoS?

Answered Question
Mar 11th, 2008

Our sites connect via routers into a single Metro Ethernet (TLS) VLAN. The main site with most servers and the Internet link uses a 100Mb TLS circuit, same for another big site; the rest are 10Mb. Majority of traffic is outbound from main site to others. There's no VoIP, limited streaming video, but occasional congestion when a large file transfer hogs a 10Mb pipe with default FIFO queue.

Where to start for QoS?

Can the main site router enforce separate 10Mb output limits for each destination, even though they share one interface?

Or should I apply input service policies to the TLS interfaces of the other routers?

Or a better way....?

Thanks much for any ideas!

Paul

I have this problem too.
0 votes
Correct Answer by Joseph W. Doherty about 8 years 9 months ago

NB: syntax likely off a bit!

ip access-list extended site1

permit ip any x.x.x.x x.x.x.x

ip access-list extended siteN

permit ip any x.x.x.x x.x.x.x

class-map site1

match access-list site1

class-map siteN

match access-list siteN

policy map yourname

class site1

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

class siteN

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

interface fastethernet0

service-policy output yourname

Shapers appear to implement FQ within themselves, so you both keep from overrunning far side and keep one flow from grabbing all the shape bandwidth.

There is a limitation to the number of classes, hopefully you don't have that many sites.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (3 ratings)
Loading.
lamav Tue, 03/11/2008 - 18:42

Paul:

I had a similar situation a while back.

Can you post a diagram?

Can you tell us more about the traffic flows?

You may have no VoIP, but what other delay-sensitive applications do you have?

I implemented a nice little QoS scheme because we were having a problem between our DR site -- which housed our servers -- and the main site whenever a data backup would be running.

pnicolette Tue, 03/11/2008 - 19:56

Hi lamav,

Diagram (not Visio):

__________________________

|.......|.......|.......|B|.......|.......|A|.....|

every site is in same VLAN, sites A & B have bigger pipes, and A originates most traffic.

Is there a config to let the outbound interface at site A do a distinct CBWFQueue for each small-pipe destination, each w/its own 10Mb ceiling?

I'm less worried about the specific apps than the queueing strategy.

lamav Tue, 03/11/2008 - 21:47

Paul:

I'm not sure what that drawing is, but it looks like something from a cave in Mesopotamia. :-)

But all kidding aside, given what you have told us, I imagine that this is a multipoint-to-multipoint (any-to-any) topology, in which all sites are in the same TELCO VLAN.

My client was running that same set up and TELCO was basically performing some dot1q tunneling.

The problem we had was that the database replication and storage/backup jobs were killing the connection between the DR site and the main site. So, what I did was classify all the mission critical traffic: VoIP, server heartbeat, and a few delay-intolerant applications, and then configured QoS to prioritize the traffic accordingly.

For example, at one site we had the following at one site:

Class-map voip

match ip dscp ef

class-map data

match access-group 100

!

policy-map TLS

class voip

set ip dscp ef

priority percent 20

class data

bandwidth percent 30

set ip dscp af31

class class-default

fair-queue

!

interface g0/0

service-policy output TLS

!

!

access-list 100 permit ip 172.16.0.0 0.0.0.255 any

..and at site 2:

class-map hb

match access-group 100

class-map data

match access-group 101

!

policy-map TLS

class hb

set ip dscp af41

class data

set ip dscp af31

!

interface g0/1

!

priority-queue out

!

service-policy input TLS

!

Access-list 100 permit tcp any any eq 580

!

Access-list 100 permit udp any any eq 580

!

Access-list 101 permit ip 172.16.0.0 0.0.1.255 any

!

[end]

So, at site 1, I prioritized the VoIP traffic by placing it in an expedited queue (ef) and alloted a certain amount of bandwidth, too. I also took the general LAN traffic data and classified it as af31 and assigned it 30 percent bandwidth.

At site 2, I identified the mission critical server heartbeat traffic and assigned it an af of 41 and the rest of the general LAN traffic was assigned as af31.

So, the methodology I used was to identify the mission critical traffic, mark and queue it accordingly, and the rest of the general user traffic would fall into a lower priority queue.

I don't know if I have helped you, but this was my experience with a TLS topology and QoS. I am not a QoS guru, but there are others on here who are, so you can stay tuned for them.

HTH

If so, kindly rate this post.

Thanks

Victor

pnicolette Wed, 03/12/2008 - 05:20

it looks like something from a cave in Mesopotamia. :-)

It should. We've got 3.5G here now!

Victor, thanks for taking the time to share your config & experience. I'm starting to learn enough QoS so it looks fine to me - though I try to use named access lists to remember what I did (gets hazy after a few millennia).

Paul

Joseph W. Doherty Wed, 03/12/2008 - 05:31

Two concerns:

Are we missing shaping?

Don't know what platform you're going to attempt this on. L3 switches may not accept a CBWFQ output policy.

Joseph W. Doherty Tue, 03/11/2008 - 20:02

Assuming you have a hub and spoke, if not physically, at least for almost all traffic flows, the QoS model you might want to start with is a 10 Mbps shaper using FQ, one dedicated for each remote site on the main site, outbound.

Whether the prior can be easily accomplished is dependent on the hardware/software platform's features and perhaps how many remotes you need to support.

pnicolette Wed, 03/12/2008 - 05:32

Joseph,

Exactly! So... 3745, 12.4T, how to configure multiple dedicated shapers?

I can ACL by destination, and mark by traffic type, but am losing my few remaining hairs trying to shape the marked traffic to 10Mb per destination.

Possible?

Correct Answer
Joseph W. Doherty Wed, 03/12/2008 - 05:51

NB: syntax likely off a bit!

ip access-list extended site1

permit ip any x.x.x.x x.x.x.x

ip access-list extended siteN

permit ip any x.x.x.x x.x.x.x

class-map site1

match access-list site1

class-map siteN

match access-list siteN

policy map yourname

class site1

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

class siteN

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

interface fastethernet0

service-policy output yourname

Shapers appear to implement FQ within themselves, so you both keep from overrunning far side and keep one flow from grabbing all the shape bandwidth.

There is a limitation to the number of classes, hopefully you don't have that many sites.

pnicolette Wed, 03/12/2008 - 06:16

NB = NP (No Problem. My IOS wants "match access-group name site1" - ugh!)

Just the FQ within shapers will be big improvement over existing. Thanks.

Now for sun AND stars. In one router, can I cascade classes or policy maps or otherwise somehow

1) mark traffic by type, AND

2) FQ it by destination?

Resulting in per-destination CBWFQ (w/WRED cherry on top).

Hmmmm....create a virtual interface that redirects (how?) to the physical interface and

do 1 on virtual and 2 on physical?

pnicolette Wed, 03/12/2008 - 06:22

or simpler...mark on input (other interfaces) and shape on output? Too obvious?? :-)

Joseph W. Doherty Wed, 03/12/2008 - 10:03

The answer is yes to both your questions, although you would restrict your inbound marking to just traffic types not traffic destinations.

The key to the outbound is usage of a heirarchical policy.

e.g. (NB: syntax likely incorrect)

class-map important

match protocol rtp audio

class map notimportant

match protocol ftp

class-map realtime

match dscp ef (might need to do this with an ACL)

class-map bulk

match dscp af11 (ditto)

policy-map classifymyway

class important

set dscp ef

class notimportant

set dscp af11

class class-default

set dscp be

policy-map queuemyway

class realtime

priority percent 30

class bulk

bandwidth percentage remaining 1

random-detect

class class-default

fair-queue

need stuff from prior post, amended:

policy map yourname

class site1

service-policy queuemyway

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

class siteN

service-policy queuemyway

shape 10000000

(might need a bandwidth statement) bandwidth percent 1

(inbound interface)

interface fastethernet 0

service-policy input classifymyway

(outbound interface)

(as prior post)

pnicolette Wed, 03/12/2008 - 12:13

Wonderful, thanks.

How do I tell if need a bandwidth statement in service-policy queuemyway? And it is percent 1 not percent remaining 1?

Joseph W. Doherty Wed, 03/12/2008 - 17:21

You'll want a bandwidth statement in queuemyway so that you can set the ratios between traffic.

Percent # is fine too. (Oh, just looked it up, the variant is "remaining percent".)

pnicolette Fri, 03/14/2008 - 08:22

Thanks again!

Finally [thanks for your patience, hope this helps others]...

What are pros/cons in my mostly hub-spoke world of using dscp designations like cs1, cs2, cs3 versus the newer, more complex scheme like ef, af31, af11, be and such?

Joseph W. Doherty Fri, 03/14/2008 - 10:14

Actually you mean the pros/cons of DSCP vs. IP Precendence?

The former provides six bits for marking vs. only 3 bits for the latter. Using RFC recommendations, priorities overlap between the two and the additional DSCP bits are used for indicating drop precedence.

If you're going to mark at all, I would recommend using DSCP. Of course, you don't need markings to deliver QoS, it's just convenient for downstream devices (so they don't have to analyse the traffic again).

mesutcap81 Wed, 01/15/2014 - 01:57

Hi Joseph,

I used your config for our qos environement which has a diagram below. Central and all branchs connected to ISP/MPLS cloud via metro ethernet lines witch different bandwidths. Central connected 100Mb, other branchs has different bandwidths.

We need QoS both central and branchs for especially voip RTP and signallling traffic. So i used your config but we have some problems.

As you see following outputs Class-map: Branch-A has 1024Kb bw and using 1024Kb bandwitdth but under Service-policy "queue",

Class-map: "realtime" and "signal" using 20000 and 5000Kb bandwidth. Actually i want to use 20% and 5% of 1024Kb.

What is the issue outthere?

Router:Cisco 3845

IOS:c3845-spservicesk9-mz.124-13f.bin

Thanks in advance.

Central_Router#sh policy-map interface g0/0

GigabitEthernet0/0

  Service-policy output: central

    Class-map: Branch-A (match-all)

      133672 packets, 93715046 bytes

      5 minute offered rate 372000 bps, drop rate 0 bps

      Match: access-group name Branch-A

      Traffic Shaping

           Target/Average   Byte   Sustain   Excess    Interval  Increment

             Rate           Limit  bits/int  bits/int  (ms)      (bytes)

        100000000/100000000 625000 2500000   2500000   25        312500

        Adapt  Queue     Packets   Bytes     Packets   Bytes     Shaping

        Active Depth                         Delayed   Delayed   Active

        -      0         133672    93715046  0         0         no

      Queueing

        Output Queue: Conversation 265

        Bandwidth 1 (%)

        Bandwidth 1024 (kbps)Max Threshold 64 (packets)

        (pkts matched/bytes matched) 454/409201

        (depth/total drops/no-buffer drops) 0/0/0

      Service-policy : queue

        Class-map: realtime (match-all)

          0 packets, 0 bytes

          5 minute offered rate 0 bps, drop rate 0 bps

          Match: ip dscp ef (46)

          Queueing

            Strict Priority

            Output Queue: Conversation 264

            Bandwidth 20 (%)

            Bandwidth 20000 (kbps) Burst 500000 (Bytes)

            (pkts matched/bytes matched) 0/0

            (total drops/bytes drops) 0/0

        Class-map: signal (match-all)

          0 packets, 0 bytes

          5 minute offered rate 0 bps, drop rate 0 bps

          Match: ip dscp af31 (26)

          Queueing

            Output Queue: Conversation 265

            Bandwidth 5 (%)

            Bandwidth 5000 (kbps)Max Threshold 64 (packets)

            (pkts matched/bytes matched) 0/0

        (depth/total drops/no-buffer drops) 0/0/0

        Class-map: class-default (match-any)

          133672 packets, 93715046 bytes

qosproblem.jpg

!---INPUT-----

!

policy-map mark

class RTP_AUDIO

  set ip dscp ef

class SIGNALLING

  set ip dscp af31

!

class-map match-any RTP_AUDIO

match access-group name rtp

!

ip access-list extended rtp

permit udp any any range 16384 32767

!

class-map match-any SIGNALLING

match ip dscp cs3

match access-group name signalling

!

ip access-list extended signalling

remark H323

permit tcp any any range 1720 1721

remark SKINNY

permit tcp any any range 2000 2002

permit udp any any range 1720 1721

remark MGCP

permit udp any any eq 2427

remark SIP

permit tcp any any eq 5060

permit udp any any eq 5060

!

!

!----OUTPUT-----

policy-map queue

class realtime

  priority percent 20

class signal

  bandwidth percent 5

class class-default

  fair-queue

!

class-map match-all realtime

match ip dscp ef

!

class-map match-all signal

match ip dscp af31

!

!---OUTPUT with BRANCH-----

!

policy-map central

class Branch-A

  shape average 100000000

  bandwidth percent 1

  service-policy queue

class Branch-B

  shape average 100000000

  bandwidth percent 1

  service-policy queue

class Branch-C

  shape average 100000000

  bandwidth percent 2

  service-policy queue

class Branch-D

  shape average 100000000

  bandwidth percent 2

  service-policy queue

!

class-map match-all Branch-A

match access-group name Branch-A

class-map match-all Branch-B

match access-group name Branch-B

class-map match-all Branch-C

match access-group name Branch-C

class-map match-all Branch-D

match access-group name Branch-D

!

ip access-list extended Branch-A

permit ip any 10.34.128.0 0.0.7.255

ip access-list extended Branch-B

permit ip any 10.34.112.0 0.0.7.255

ip access-list extended Branch-C

permit ip any 10.34.152.0 0.0.7.255

ip access-list extended Branch-D

permit ip any 10.35.144.0 0.0.7.255

!

interface GigabitEthernet0/0

bandwidth 102400

ip address 172.100.0.2 255.255.255.248

service-policy output central

!

interface GigabitEthernet0/1

ip address 172.18.0.2 255.255.255.252

service-policy input mark

!

Actions

This Discussion