Difficult QOS problem

[emelville]

I have a 500 node MPLS network. All the remote routers are 2811's and all the remote traffic terminates here at the central site on a 7606 with a DS3. We have implemented service policies in all the remote routers and they seem to be working well. However, we are "slamming" or overrunning the remote circuits because we are not shaping our traffic down by remote site as it leaves our DS3. My understanding is that you can nest policies, but that is a huge amount of ACL's and policies. I'm not even sure it would work.

Any advice or recommendations would be appreciated.

[Dougblock]

We are just running across the MPLS Cloud and not doing LSR. Our VOIP and another key system are hub and spoke design in a MPLS network.

[Pavel Bykov]

We have the similar setup as you - 220 nodes with variable speeds. Hub site is 200 Mbps, and end nodes are from 128Kbps to 1024 Kbps.

And we have a working solution - All QoS is done on both sides of the bottle neck. In every case the bottleneck is the slow line of the remote.

E.g. If it's:

central CPE <-> CE <-> PE <-> CLOUD <-> PE <-> CE <-> CPE remote

The bottleneck is PE remote <-> CE remote, the rest of the network is High Speed. We have QoS implemented on output of PE remote to CE remote, and on output from CE remote to PE remote.

You need to implement QoS where your software queues are created. One way is to manually create those queues using Shaping at central site, but that's not scalable or easily possible, as you have mentioned. Therefore you need to implement it on every device that has the slow link connected.

Hope this helps.

[Dougblock]

we have tried a simple test like that and did get some good results but we have only a 2-DS3 as our connection to the MPLS cloud which is also a bit over subscribed which is the main problem. We don't have a problem accing MQC to the remote nodes out there but the Hub site is going to be a bit harder.

[Pavel Bykov]

Well, but hub is almost a no issue.

If you have done QoS on ALL of your branch lines, there is not much point to do the QoS at the central site. Just trace the speeds and see where the queues will form. Sure there will be some at output of hub to branches, but they are many magnitudes smaller than what will be formed at the branch side.

[emelville]

I don't understand why you don't recommend using QOS at the central site. You need to tag the outbound packets from the hub to ensure reliable deliver to the remotes. Otherwise, everything would be seen as "default" and subject to whatever the carrier chose to do with it. This would be especially true for voice traffic- it would not receive jitter-free handling.

[Joseph W. Doherty]

I agree with Pavlo, best place to initially place QoS is where the bottlenecks are.

Reason I asked about whether your routers were LSRs, is trying to understand whether you also control/set QoS policy on MPLS egress. If not, then without getting into the problem of managing some type of shaping at the central site, and if your carrier offers MPLS egress QoS policies, mark your outbound packets at the central site to take advantage of them. (Marking packets at the hub outbound, I would consider part of WAN end-to-end QoS.)

Another possibility, I don't recall whether there any options to shape, police or set policies on tunnels, but if there were, you could control the outbound traffic via a tunnel.

[Pavel Bykov]

Let me clarify myself. Queues formed at central site are negligible since it is not a primary (or even secondary) bottleneck.. Classification and Marking features of QoS besides congestion management and avoidance are almost mandatory at central HUB. Because first output of a router with a slow link should use DSCP values for classification.

So, QoS at central hub: Definitely. But congestion management and Congestion Avoidance will have little to no effect. I still stringly recommend implementing it for times of high load to multiple branches at once.

Also, if MPLS network is under your control, remember that three high order bits of DSCP field are automatically copied into MPLS EXP field. If you are controlling MPLS routers, you can setup congestion management and avoidance on those routers based on MPLS EXP bit. But again - concentrate on your bottlenecks.

It is not that hard to configure 2x500 routers (from both sides of a slow link) as you might think - Same policy map (that uses percentage rather than absolute values) came be applied on all of them. And if you (or your provider) are using same interface numbers on branches, it can even be pasted over SNMP or some other automation tool.

[Pavel Bykov]

I thought my post got lost, but then I found out that it was just moved to a next page. Long thread I guess :)