How to rate-limit different IP's entering/leaving an Interface?

Unanswered Question
May 7th, 2009

Hi There,

We are an ISP and have an interconnect with say Provider A. Customers of ours use Provider A for layer 2 and us for Layer 3 (IP/Internet).

Customer #1 to #100 --- (10Mb) --> Provider A ---> ISP ---> Internet

We'd like to rate limit some customers to 2mb/sec (in/out) on our end because at present they have a 10mb/sec connection coming from Provider A.

The config we use to peer with Provider A is this:

interface GigabitEthernet0/1.120

description Interconnect with Provider A

bandwidth 400000

encapsulation dot1Q 120

ip address A.B.C.1 255.255.255.252

Customers of ours are simply routed out through this interface like so:

Customer #1:

ip route W.X.Y.Z. 255.255.255.255 A.B.C.2

Customer #2:

ip route J.K.L.M 255.255.255.255 A.B.C.2

Is there a way to rate limit both these customers without needing to create a separate class map for each like so:

class-map match-all CUSTOMER-1-2MB

match access-group name ACL-CUSTOMER-1

class-map match-all CUSTOMER-2-2MB

match access-group name ACL-CUSTOMER-2

!

policy-map POLICY-RATE-LIMIT

class CUSTOMER-1-2MB

police 2000000 375000 750000

class CUSTOMER-2-2MB

police 2000000 375000 750000

!

interface GigabitEthernet0/1.120

service-policy input POLICY-RATE-LIMIT

service-policy output POLICY-RATE-LIMIT

I've done this in the lab and I know it works, so Customer #1 ends up with 2mb/sec and Customer #2 gets 2mb/sec too.

But in production, I'm talking about 100's of customers which we simply route out of this interface. I can't imagine having to configure 100's of class maps and policy maps for each customer to rate limit them to 2mb/sec like in the config above.

Is there a better way to do this on the router???

Thanks.

Andy

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.1 (5 ratings)
Loading.
Collin Clark Fri, 05/08/2009 - 08:06

Change

<font size="2"></p><p>class-map match-all CUSTOMER-1-2MB</p><p>match access-group name ACL-CUSTOMER-1 </p><p></font>

to

<font size="2"></p><p>class-map match-<b>any</b> CUSTOMERS-2MB</p><p>match access-group name ACL-CUSTOMERS-2MB</p><p></font>

The ACL-CUSTOMERS-2MB ACL would then have each subnet of your customers you want to police. You can then create class-maps by speed rather than one for each customer.

Hope that helps.

asaykao73 Sun, 05/10/2009 - 17:24

Hi Collin,

Thanks for your reply.

I don't think this way will work as I've labbed it up and the customers in each separate ACL share the same bandwidth. We are trying to make it so that each customer get their own 2mb bucket.

If we had multiple ACL's in the same class map, those customers would all end up sharing the 2mb bucket and not have their own individual 2mb bucket.

Cheers.

Andy

Laurent Aubert Sun, 05/10/2009 - 12:00

Hi Andy, Collin,

Collin, I think with your configuration, you will rate-limit all the customers to 2Mbps instead of each customer to 2Mbps.

Andy, even if you want to apply the same policer to all your customers, it's still seen as a policer per customer so you need to be able to differentiate each customers. Having a class-map per customer matching a dedicated ACL is the only way I know to do what you want to do.

HTH

Laurent.

asaykao73 Sun, 05/10/2009 - 17:21

Hi Laurant,

Thank you for your reply.

I believe you are correct. In production, we made the mistake by doing it the way Collin suggested (oops) but when I went to lab it up, I found it didn't work that way, instead all the customers under the same class map (no matter if they were in the same ACL or in a different ACL), all ended up sharing the 2mb bandwidth.

Some people suggested I try using ISG but the docos are pretty bad and I didn't really understand it.

Configuring ISG Access for IP Subscriber Sessions

http://www.cisco.com/en/US/docs/ios/12_2sb/isg/configuration/guide/isb_ip.html

If somebody has any ideas on how to make it work using ISG, I'll be happy to know about it :)

Thanks.

Andy

Collin Clark Mon, 05/11/2009 - 05:26

Laurant/Andy-

You're right, that config would restrict them all instead of each. Sorry for the incorrect information.

asaykao73 Thu, 05/14/2009 - 17:56

Hi All,

Doing what Laurent suggested works great. We are able to rate-limit the desired customers by having a separate class-map for each customer that needs to be rate limited under the single policy-map.

But now we're finding that the CPU utilization has increased by 20% because of this.

We're pushing about 400M through this interface and rate limiting 7 customers on it.

Given the large traffic flow through this interface (400M), is it common to see an increase in CPU utilisation by 20%???

When we take the service-policy off the interface, sure enough the CPU drops by 20%.

Here's the MQC applied:

interface GigabitEthernet0/1.120

bandwidth 400000

encapsulation dot1Q 120

ip address 203.17.x.x 255.255.255.252

ip flow ingress

service-policy input RATE-LIMIT

service-policy output RATE-LIMIT

!

class-map match-all CLASS-TCS-200338

description Customer #1 rate limited to 4mb/sec

match access-group name ACL-TCS-200338

class-map match-all CLASS-TCS-200208

description Customer #2 rate limited to 2mb/sec

match access-group name ACL-TCS-200208

class-map match-all CLASS-TCS-205593

description Customer #3 rate limited to 3mb/sec

match access-group name ACL-TCS-205593

class-map match-all CLASS-TCS-205679

description Customer #4 rate limited to 4mb/sec

match access-group name ACL-TCS-205679

class-map match-all CLASS-TCS-200441

description Customer #5 rate limited to 4mb/sec

match access-group name ACL-TCS-200441

class-map match-all CLASS-TCS-200005

description Customer #6 rate limited to 2mb/sec

match access-group name ACL-TCS-200005

class-map match-all CLASS-TCS-205560

description Customer #7 rate limited to 4mb/sec

match access-group name ACL-TCS-205560

!

policy-map RATE-LIMIT

class CLASS-TCS-200005

police 2000000 375000 750000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-200208

police 2000000 375000 750000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-200441

police 4000000 750000 1500000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-200338

police 4000000 750000 1500000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-205679

police 4000000 750000 1500000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-205560

police 4000000 750000 1500000 conform-action transmit exceed-action transmit violate-action drop

class CLASS-TCS-205593

police 3000000 562500 1125000 conform-action transmit exceed-action transmit violate-action drop

Is this the correct behaviour of applying the service-policy to the interface whereby the CPU increases by 20% or can the MQC be fine tuned to have less of an impact on the CPU?

Is the router just overloaded, taking into account it's only pushing about 400M? Should it be able to do more than this??

Thanks.

Andy

asaykao73 Thu, 05/14/2009 - 18:32

The router we're using is:

Cisco 7206VXR (NPE-G1) processor (revision A) with 983040K/65536K bytes of memory.

Processor board ID 29741318

SB-1 CPU at 700Mhz, Implementation 0x401, Rev 0.2, 512KB L2 Cache

6 slot VXR midplane, Version 2.7

Cisco IOS Software, 7200 Software (C7200-A3JK91S-M), Version 12.2(31)SB13, RELEASE SOFTWARE (fc1)

Laurent Aubert Fri, 05/15/2009 - 06:29

Hi,

7206 is a software platform so it's expected to see the CPU increasing with a large QoS configuration.

HTH

Laurent.

Collin Clark Fri, 05/15/2009 - 06:34

If I remember correctly, routers perform QoS in software and switches in hardware. Is that correct Laurent?

Laurent Aubert Fri, 05/15/2009 - 07:36

QoS is handle in software from 8xx to 7200 chassis and in hw for upper platform (ASR1000, 12000,..).

QoS is in hw on switches but the implementation is different from routers one

Laurent.

Joseph W. Doherty Sun, 05/17/2009 - 03:24

As the others have noted, IOS features on a software based router are performed by the CPU, so it's not totally unexpected the policy policing adds to the CPU load.

What might help is trying configurations that could use a more efficient code path.

No guarantees, but you might insure/try:

o Turbo ACL is active

o flow cache on all interfaces is active

o change match-all to match-any

o use the rate-limit rather than CBWFQ police

asaykao73 Sun, 05/17/2009 - 16:56

Thanks for everybody's reply.

I re-arranged some class-maps and ACL's around and got much better CPU performance.

Rather than having the majority of packets entering and leaving the interface scroll through each class-map before hitting the class-default class, I created the very first class map to "match not" against the ACL's I was using for each customer.

1/ Define a class-map to match all customers that are NOT to be rate-limited:

class-map match-all DEFAULT-TRAFFIC

description Match all customers that are NOT to be rate limited

match not access-group name ACL-TCS-200338

match not access-group name ACL-TCS-200208

match not access-group name ACL-TCS-205593

match not access-group name ACL-TCS-205679

match not access-group name ACL-TCS-200441

match not access-group name ACL-TCS-200005

match not access-group name ACL-TCS-205560

2/ Apply this new class-map as the FIRST class to be looked at inside the policy-map.

policy-map DEFAULT-RATE-LIMIT

class DEFAULT-TRAFFIC

class CLASS-TCS-200005

police 2000000 375000 750000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-200208

police 2000000 375000 750000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-200441

police 4000000 750000 1500000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-200338

police 4000000 750000 1500000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-205679

police 4000000 750000 1500000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-205560

police 4000000 750000 1500000 conform-action transmit

exceed-action transmit violate-action drop

class CLASS-TCS-205593

police 3000000 562500 1125000 conform-action transmit

exceed-action transmit violate-action drop

Doing it this way didn't have any impact on the CPU and surprised me somewhat.

I then googled around and found the importance of structuring your ACL's to get better CPU performance, so I redid the ACL's under class-map DEFAULT-TRAFFIC and instead of having a separate ACL per customer to "match not" with, I put all of the customers into one big ACL.

ip access-list extended ACL-DEFAULT

remark ACL-TCS-200338

permit ip host x.x.x.

remark ACL-TCS-200208

permit ip host x.x.x.x

remark ACL-TCS-205593

permit ip host x.x.x.x

remark ACL-TCS-205679

permit ip host x.x.x.x

remark ACL-TCS-200005

permit ip host x.x.x.x

remark ACL-TCS-205560

permit ip host x.x.x.x

!

class-map match-all DEFAULT-TRAFFIC

match not access-group name ACL-DEFAULT

Doing it this way caused a reduction in CPU by 10%.

Thanks.

Andy

Joseph W. Doherty Sun, 05/17/2009 - 18:00

The good news, structuring logic to match expected conditionally frequencies can have a huge impact on performance. The bad news, frequencies can change. From a maintenance viewpoint, good to avoid, if possible.

For example, if "A" occurs 15%, "B" occurs 5%, and "C" occurs 80%, an "or" condition often would work best "if C or A or B" rather than "if A or B or C". However, now we've tied our logic to the expectation of A vs. B vs. C occurences. (NB: BTW, I'm not saying this is bad, but just be aware of all the implications.)

Also, it wouldn't matter on your input policy, but by moving "default" traffic to a defined class, you've might have also moved that traffic from a (default) FQ to FIFO.

asaykao73 Sun, 05/17/2009 - 18:50

Thanks Joseph.

I will definitely keep that in mind.

Much appreciate your input.

Cheers.

Andy

n-magbade Wed, 09/23/2009 - 07:40

Would the police statement be different if you have say 2mb download and 1mb upload

asaykao73 Wed, 09/23/2009 - 16:53

Yes you will need a separate service policy for the input and output on your interface.

Eg:

interface GigabitEthernet0/1.120

service-policy input MY-POLICY-1MB

service-policy output MY-POLICY-2MB

Cheers.

Andy

chrmar Wed, 10/14/2009 - 22:11

ISG has several options for doing this in an elegant fashion. See "Configuring ISG Policies for Regulating Network Access." Policing can be applied either per user, or per traffic class. Additionally it can be done via locally configured service-profile, aaa user profile or aaa service profile.

If you require some basic configuration for ISG (not easy to divine from the documentation), I can provide this.

Safar Safarov Mon, 08/06/2012 - 13:14

Hi Andy,

did you find any better way how to do this? I have around 60 customers and it's really a pain to manage them in the way described (policy per customer).

Regards,

Safar.

Actions

This Discussion