PfR symmetry

[philip moore]

Dear All,

I am mid way through delivery of an MPLS network for a customer. We have the option of adding a DMVPN internet backup. The WAN routing protocol is BGP for both DMVPN and MPLS (MPLS is straight routing - no overlay or getvpn)

The customer has some pretty odd ideas about how to use the service, mainly they are very paranoid about oversubscibing the MPLS link for their "golden" application, so have come up with all kinds of weird and wacky solutions that they call "requirements". As a result, they have the idea that they cannot route all traffic over MPLS, and instead want to create a mix of route filter, PBR, NAT, ACL and all kinds of horrible things to keep the MPLS link "sacred".

I would like to guide them to use the service in a better way leveraging PfR, something simple to start with like:

- use MPLS all of the time by default

- when it reaches 80% utilisation, switch the non-critical traffic to DMVPN yet keep the "golden" app on MPLS

- this policy is applied at both branch and hub BRs

I have read this cisco wiki (it is rather good!!!!):

http://docwiki.cisco.com/wiki/PfR:Solutions:EnterpriseDualVPN#PfR_performance_and_load_policy_test_case

Where I get a bit lost is in the symmetry. Is it symmetrical?

For example, if a branch MPLS link hits 81% outbound utilisation, then uses dynamic PBR to switch outbound traffic to VPN. Sorted! but what about the inbound? If that is hitting 81% also. What does PfR do?

This is where I get a bit lost. For the inbound to branch, the hub must do something surely? Consider the MPLS capacity is 100M - the 8.1M to the branch is in policy so should keep forwarding via MPLS. How does it know that the branch is OOP?

If it doesn't do that, you end up with assymetric traffic (branch outbound DMVPN, inbound MPLS).

Am I missing something here? Any advice you can give us much appreciated.

Thanks a million! 

Phil

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

If the customer has a "golden" application, then they and you might want to first investigate QoS to protect it, whether MPLS or DVMPN.

Once you've have a QoS model, something like PfR can be used to reroute traffic in conjunction with QoS.  As your reference mentions, PfR is great for dealing with "brownouts" and "blackouts" within a cloud (assuming you have alternative path[s]).  However, I don't agree with your reference to move traffic about based on a simple rule like exceeding 80% utilization, nor do I agree waiting until traffic loss is "moderate" (defined at >= 20%!!!).

Regarding your questions about symmetry, no PfR isn't.  So, yes, PfR can create asymmetric traffic flows, and your concern for this is because?

Regarding your concern about inbound bandwidth management, actually inbound is indirectly managed if the other side's outbound is managed, but PfR does have direct inbound management capabilities, but it requires "cooperation" with the sending routers.

Regarding your question how does PfR HQ know a BR is OOP?  Well, it sees a jump in latency, at the HQ, for traffic to that BR.

PS:

BTW, my experience has been Internet VPN, with technologies like QoS and PfR, can perform similar to MPLS (FR or ATM) clouds.

[philip moore]

Hi Joseph

Thanks so much for your reply.

First I will add the missing info, yes there is QoS policy on both MPLS (5-class model with voice, video + 3 data classes, I don't think they are shaping, I expect is police.) and similar policy for VPN (using per-tunnel QoS). They both share the same DSCP marking, marked in the LAN ingress to the BR. SO yes, PfR traffic class by DSCP marking is what I would be looking into.

Asymmetric is seen as a bad word in the customer's world, and is something to be avoided. Usually it is because it means some routing has gone awry and possibly traffic is being routed one way via high speed leased-line, and back via a low bw/high latency VSAT. Also apps like Domino don't seem to like it very much and refuse to transfer mail under those conditions. But all that said, I can try it.

I think PfR can help us with the outbound traffic from branch, but I struggle to see how it can help with the inbound IF using utilization only on HUB BR - agree it would never go out of policy (10M to a single branch means the hub is only at 10%).

So to enable this you think I should be looking at monitor delay also at HUB?

Or have you any experience with inbound route control using BGP?

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/pfr/configuration/xe-3s/pfr-xe-3s-book/pfr-bgp-inbound.html

The MPLS WAN does not use GETVPN. Do you think using that gives better options for traffic control?

The suggested PfR policy came straight from the Cisco example. What is typical PfR policy for you/in the real world?

Thanks a lot for your interest!

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Ah, well my experience with asymmetrical routing has often been with similar bandwidth paths, so it's not normally a problem unless there's some "stateful" device between hosts.  But I can see why inadvertent transit via VSAT might be considered a problem.  ;)

I think PfR can help us with the outbound traffic from branch, but I struggle to see how it can help with the inbound IF using utilization only on HUB BR - agree it would never go out of policy (10M to a single branch means the hub is only at 10%).

Ah, it works because, as noted, PfR will see the jump in latency (passively and/or actively) to the destination if you have it do more than monitor egress port utilization.  (When I had used OER/PfR, I had international multipoint.  So "in" congestion to any one site could be caused by the aggregate of multiple other sites "out" to it.

With active monitoring, PfR would see the MPLS link latency worse than your VPN backup (to same destination), and when it does, juggle traffic flows.

No, don't have experience with using PfR's inbound control (at the time I set it up, most of my devices only supported OER, and inbound isn't an option with OER).

Also cannot say what's typical "real world". I can say, when I setup "my" OER/PfR, our network monitoring section complained almost all our WAN performance problems disappeared.  (Laugh - we then had to work out how to "see" WAN performance problems in spite of OER/PfR.)