Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

Understanding of Policy Based routing

Hi,

I have a scenario for Policy based routing and I would like to have a opinion how the traffic should flow.

PBR is active on the LAN and is filtering traffic with the help of route-map which refers to a ACL. (standard configuration)

The route-map has a recursive next hop of it's neighbor which is not a direct hop but known via BGP.

Suppose that this recursive next hop is not seen anymore in BGP routing table, how should the router behave?

a) The packets are matched no matter if the recursive next hop is seen or not and is routed out over the PBR WAN interface.

b) The packets are matched even if the recursive next hop is not seen, but it sends the packets over the default WAN interface.

c) The packets are not matched due to the fact that recursive next hop is not seen in the BGP table and is routed directly over the default WAN interface.

Looking forward to hearing from someone what the right scenario should be.

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Understanding of Policy Based routing

Hello

I hope I can explain this too you and at the same time give it some justice, if not I am sure someone from this forum will correct me.

Policy routing as I see it is a " posh form of static routing, where as static routing is via per destination only policy routing can be source and destination or other forms of matching value to name a few to force traffic to a different path other than  the routing protocols are dictating.

Standard acl - source address only

Exceeded acl's - source and destination

Metrics value

Packet lengths

You can manipulate traffic based on some  kind of POLICY you set in route-maps  to effect traffic

Originating from the router - local policy routing - Control Plane

Traversing the router from a particular interface. -  Data Plane

Now to answer you question regards recursive next hop - if the destination specified isn't available I am on the understanding it will route via default route in the rib unless otherwise stated say via a set interface null 0 value.

Please see below is taken from  txt file I have complied mostly plagiarized from either cco  documentation or some other routing text book.

Policy routing:
###############

PBR can contain multiple set commands within a single route-map sequence number.
If multiple set commands are configured, the order that PBR will attempt them in is:


1. set ip next-hop verify-availability  – with object tracking only

2. set ip next-hop - either alone or with CDP verification

3. set ip next-hop recursive

4. set interface

5. Route table

6. set ip default next-hop - either alone or with CDP verification

7. set default interface


Recursive Next-Hop
######################
set ip next-hop recursive allow multiple interfaces or next-hop addresses to be listed in a single set statement.
If the 1st interface (or interface associated with the next-hop address, if a next-hop is used) is down or L2 address is unknown,
the rest of the interfaces/addresses in the list are tried in order

The infrastructure provided by CEF or process switching
performs the recursion to the next-hop IP address. The configuration sequence, which affects routing, is as follows:

1. Next-hop = Attempts PBR first

2. Next-hop recursive = Attempts PBR first

3. Interface = Attempts PBR first

4. Default next-hop = checks rib first

5. Default interface = check rib first


If both a next-hop and a recursive next-hop IP address are present in the same route-map entry,
the next-hop is used.

If the next-hop is not available, the recursive next-hop is used.

If the recursive next-hop is not available and no other IP address is present,
the packet is routed using the default routing table; it is not dropped.

If the packet is supposed to be dropped, use the set next-hop recursive command followed by a
set interface null0 configuration.

Hope this helps

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

Please don't forget to rate any posts that have been helpful. Thanks.
Purple

Understanding of Policy Based routing

Hi,

I suppose you have multiple outgoing interfaces ? Have you got 2 paths installed in the RIB and are you doing load-sharing ?

Regards

Alain

Don't forget to rate helpful posts.

Don't forget to rate helpful posts.
19 REPLIES
Purple

Understanding of Policy Based routing

Hi,

The answer is b)

Regards

Alain

Don't forget to rate helpful posts.

Don't forget to rate helpful posts.
New Member

Re: Understanding of Policy Based routing

Hi Alain,

That's what I would have answered too, but I have placed test ACL's on the outgoing default WAN interface and I don't see the traffic passing by

lincol01-nm#sh ip access-lists TESTdefault-out

Extended IP access list TESTdefault-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www  <== No matches

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (781301895 matches)

Extended IP access list TESTOFFLOAD-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www (393 matches)

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080 (206 matches)

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (241177730 matches)

Purple

Understanding of Policy Based routing

Hi,

I suppose you have multiple outgoing interfaces ? Have you got 2 paths installed in the RIB and are you doing load-sharing ?

Regards

Alain

Don't forget to rate helpful posts.

Don't forget to rate helpful posts.
New Member

Re: Understanding of Policy Based routing

Hi Alain,

Well the routes concerning the actual offload ACL is known via the offloading and the default WAN interfaces (BGP)

The only thing which is learned extra from the offloading interface are the recursive next-hop addressess.

New Member

Understanding of Policy Based routing

Hi Paul,

I have reviewed your answer and it turns out that the right answer was also listed in your reply!

"Now to answer you question regards recursive next hop - if the destination specified isn't available I am on the understanding it will route via default route in the rib unless otherwise stated say via a set interface null 0 value"

thanks for your posts as well as Alain!

Re: Understanding of Policy Based routing

Hello

I hope I can explain this too you and at the same time give it some justice, if not I am sure someone from this forum will correct me.

Policy routing as I see it is a " posh form of static routing, where as static routing is via per destination only policy routing can be source and destination or other forms of matching value to name a few to force traffic to a different path other than  the routing protocols are dictating.

Standard acl - source address only

Exceeded acl's - source and destination

Metrics value

Packet lengths

You can manipulate traffic based on some  kind of POLICY you set in route-maps  to effect traffic

Originating from the router - local policy routing - Control Plane

Traversing the router from a particular interface. -  Data Plane

Now to answer you question regards recursive next hop - if the destination specified isn't available I am on the understanding it will route via default route in the rib unless otherwise stated say via a set interface null 0 value.

Please see below is taken from  txt file I have complied mostly plagiarized from either cco  documentation or some other routing text book.

Policy routing:
###############

PBR can contain multiple set commands within a single route-map sequence number.
If multiple set commands are configured, the order that PBR will attempt them in is:


1. set ip next-hop verify-availability  – with object tracking only

2. set ip next-hop - either alone or with CDP verification

3. set ip next-hop recursive

4. set interface

5. Route table

6. set ip default next-hop - either alone or with CDP verification

7. set default interface


Recursive Next-Hop
######################
set ip next-hop recursive allow multiple interfaces or next-hop addresses to be listed in a single set statement.
If the 1st interface (or interface associated with the next-hop address, if a next-hop is used) is down or L2 address is unknown,
the rest of the interfaces/addresses in the list are tried in order

The infrastructure provided by CEF or process switching
performs the recursion to the next-hop IP address. The configuration sequence, which affects routing, is as follows:

1. Next-hop = Attempts PBR first

2. Next-hop recursive = Attempts PBR first

3. Interface = Attempts PBR first

4. Default next-hop = checks rib first

5. Default interface = check rib first


If both a next-hop and a recursive next-hop IP address are present in the same route-map entry,
the next-hop is used.

If the next-hop is not available, the recursive next-hop is used.

If the recursive next-hop is not available and no other IP address is present,
the packet is routed using the default routing table; it is not dropped.

If the packet is supposed to be dropped, use the set next-hop recursive command followed by a
set interface null0 configuration.

Hope this helps

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Re: Understanding of Policy Based routing

Hi Paul,

Thank you for you're extensive mail what do you mean with:

"If the recursive next-hop is not available and no other IP address is present,
the packet is routed using the default routing table; it is not dropped."

Because I have for each remote site a seperate route-map sequence which has it's own recursive next hop.

I would assume that when ever the recursive next hop statmenet for that particular route-map sequence is not available it would sent the traffic over the default WAN interface which doesn't seem to be the case as shown in the reply to Alain.

Purple

Understanding of Policy Based routing

Hi,

I have tested it with only one route-map statement with recursive next-hop, I shut down the interface which was the next-hop and here is what I got:

Mar  1 04:35:55.830: IP: s=1.1.1.1 (local), d=22.22.22.22, len 100, policy match

*Mar  1 04:35:55.834: IP: route map test, item 10, permit

*Mar  1 04:35:55.834: IP: s=1.1.1.1 (local), d=22.22.22.22, len 100, policy rejected -- normal forwarding.

Regards

Alain

Don't forget to rate helpful posts.

Don't forget to rate helpful posts.
New Member

Re: Understanding of Policy Based routing

Hi Alain,

Yes so we can conclude that the traffic is going always through the  route-map of PBR no matter if the PBR link is valid or not meaning that the recursive-next-hop seen or not....

And now for the next step where does the traffic go to?

Should you expect the traffic to go over the default WAN interface or PBR WAN interface?

This is what I got during my test.....

1) I can see my traffic is going to the PBR ACL which is set on my inbound LAN interface

nm#clear ip access-list counters

nm#sh ip access-lists traffic-offload-acl-Cher

Extended IP access list traffic-offload-acl-Cher

    7 permit tcp host 172.25.32.194 host 172.27.96.10 eq www (381 matches)

    8 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    9 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

2)  When performing a debug  I see the following output on that same router

nm#debug ip policy dynamic

Dynamic PBR debugging is on

nm#ter mon

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

and sometimes I see the following but the bottom line is that my ping test are not succesfull! when disabling the PBR link on the other end....

Nov 12 19:36:52.305 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.305 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match, len 40, FIB policy rejected(no match) - normal forwarding

3) Also when I tried to find out to which WAN interface my packets were leading,  I didn't see any packet increase on both the outbound ACL of my WAN interfaces....

nm#sh ip access-lists TESTOFFLOAD-out

Extended IP access list TESTOFFLOAD-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (747 matches)

nm#sh ip access-lists TESTdefault-out

Extended IP access list TESTdefault-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (17496 matches)

So I wonder where my traffic is going since I don't see anything increase are you able to verify if you actually see your traffic going over the WAN interface?

Re: Understanding of Policy Based routing


Hello

Then I would expect route would try to go via the next hop route path dictated by the routing protocol and then the next-hop interface.

So in this case it looks like Cadet Alain post is from 2-4 in the below sequence.

1. Next-hop = Attempts PBR first

2. Next-hop recursive = Attempts PBR first

3. Interface = Attempts PBR first

4. Default next-hop = checks rib first

5. Default interface = check rib first

what's your routing table showing regards the next hop for these routes?

Paul

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Re: Understanding of Policy Based routing

Hi Paul,

Well my routing table showing nothing once the BGP is timed out.

Maybe I should go 1 step back in order to clear out something.

A few monthns ago I also had opened a topic that PBR is not working continously as it should or missing packets,  maybe you can check my history posts for that in order to clear out the situation for you...

https://supportforums.cisco.com/message/3985581#3985581

So during that time I found out that my recursive-next-hop was learned via the Default as well as the PBR WAN interfaces and when I tried to check the routing table for that specific recurrsive-next-hop route I noticed it continiously was resetting, a normal route would have stay stable for at least couple of hours but these routes where reset every 10 minutes or something like that.  Then I thought maybe I should block my recursive next hop addresses to go out the default WAN interfaces. And that solved the problem but I think maybe blocking the next-hop routes are working now against me...or maybe it's just coincidence

New Member

Re: Understanding of Policy Based routing

just a sidenote before my last change 5months ago, the PBR mechanism went flawless!

Re: Understanding of Policy Based routing

Hello

It seems to me reading you posts that the root cause of this issue is the reclusive next hop address and how the router is seeing it valid path towards it, however saying that you say this what a stable setup 5 months back and after some amendments you started having issues.

What were the additional rules you implemented?

Have you tried specifying multiple next hop ip addresses instead and set the verify-availability option also to track these next hop address for added stability?

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Re: Understanding of Policy Based routing

Hi Paul,

I am not sure what you mean with the root cause or which root cause you refer to?

The only thing what I did 5months ago in order to stabilize the PBR mechanism (meaning letting it work continuously instead occasionally) is adding a route-map in BGP, in order to prevent the recursive-next-hop to be advertised over the default WAN interface. Which made sure that recursive-next-hops to stay stable.

And before the change I also used recursive-next-hop

The reason why I use recursive-next-hop, is because my next-hop neighbor is not directly adjacent to me. So basically the recursive-next-hop is making sure that 2 routers are aware of each other even if they are not directly connected with each other.

next-hop can only be used if your neighbor is in the same subnet which is not the case in my situation and the only option is to use recursive-next-hop.

If my next-hop would be directly adjacent to me than I would use next-hop instead, at least that is my understanding between those 2 forms of next hop.

And yes I even tried reachability tracking and insertion of a default route but nothing helped here as well.

               ---------------

router1<                                         >router2

               -----------------

Re: Understanding of Policy Based routing

Hello

Are you aware of any uRPF enabled  anywhere validating the soruce addrressing?

res

Paul

Please don't forget to rate any posts that have been helpful.for verification of the socurce 

Thanks.

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Re: Understanding of Policy Based routing

Hi Paul,

I have searched in the TFTPboot for the following syntax "verify unicast" and I didn't find any router with that particular syntax.

So should it be present or not? Since I am not very familiar with this feature

Re: Understanding of Policy Based routing

Hello

I was querying if this was applied?
Basically depending on the mode its set in can prohibit the connection from a source .

What you can apply if it isn't already is under the route-map - set ip next-hop verify-availability

This may assist in verifying the validity of the recurcursive IP address

Res
Paul

Sent from Cisco Technical Support iPad App

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Re: Understanding of Policy Based routing

Hi Paul,

I have looked in my mail and I found out I already have tried your suggestion.

So again just to summarize.

1) I have already used verify availability in the route-map as shown below

In fact the debug of the router even showed that the normal forwarding happened but my ping test failed

maybe I should perform some additional test and use a bidirectional configuration.

route-map traffic-offload permit 10

description Offloadingv2-via-Nij

match ip address traffic-offload-acl

set ip next-hop verify-availability 172.25.100.209 100 track 1

route-map traffic-offload permit 20

description Offloadingv2-via-Oud

match ip address traffic-offload-acl

set ip next-hop verify-availability 172.25.100.217 200 track 2

ip sla monitor 1

type echo protocol ipIcmpEcho 172.25.100.209 source-interface FastEthernet0/1

frequency 10

ip sla monitor schedule 1 life forever start-time now

ip sla monitor 2

type echo protocol ipIcmpEcho 172.25.100.217 source-interface FastEthernet0/1

frequency 10

ip sla monitor schedule 2 life forever start-time now

2)  I also used a default route statement in the route-map as shown below and come to think of I probably didn't used a configuration on the other end.

route-map traffic-offload permit 10

description Offloadingv2-via-Nijmegen

match ip address traffic-offload-acl

set ip next-hop recursive 172.25.100.209

set ip default next-hop 134.222.146.174

route-map traffic-offload permit 20

description Offloadingv2-via-Oudemeer

match ip address traffic-offload-acl

set ip next-hop recursive 172.25.100.217

set ip default next-hop 134.222.146.174

Bottom line, at this moment I am not sure what happened to the packets since the outgoing ACL WAN interface output are not listed  so I have to go back to the drawing board and make a new test plan and perform the 2 steps above again. I will keep you posted about the progress

New Member

Re: Understanding of Policy Based routing

Dear All,

I finally managed to pinpoint the problem here!

and the reason why it didn't work anymore was, that the customer was advertising a default route from the LAN, meaning the offload traffic was redirected back towards the LAN instead of the WAN interface!

Despite the fact that I had a default interface statement in my route-map!

    ••>Packets where travelling this way as it should =••>====••>

                        ---------------

router1 remote<                                         >router2 HQ   <••Reply packets received from the customer<••==<••

                      -----------------

So just to bring back the last scenario: I couldn’t remember what happened to the reply packets on the HQ! Since I didn’t had any information of the test ACL outputs!

Anyway so I was testing the situation again with the customer last evening and I found out that offload packets from the remote site was sent over the default interface after I had disabled the offload tunnel interface of the remote site. (which is good)

Then I checked on the HQ, if the packets where offered towards the LAN which went just fine as well!

However the reply packets where not seen over the WAN interfaces! (default WAN interface & offload WAN interface)

And I verified once again if I received any reply packets from the customer on the LAN interface which seemed to be the case!

So then I was thinking where could the packets lead to if it’s not been seen over the WAN interfaces!

The suspicion was that maybe the default route which was advertised by the customer could be the problem!?

And I made an incoming route-map on the BGP session with the customer, blocking the default route from the customer and immediately the traffic started to go over the default WAN interface as it should!

What have I learned from the situation?

As soon as the packet is processed by a PBR interface it doesn’t look to the actual destination route!

No!, instead the router is looking for the recursive-next-hop address which is not existing, so it tries to find an alternative route which is the default route!

Apparently the default route is more attractive than the PBR default interface statement!

route-map traffic-offload permit 50

match ip address traffic-offload-acl-Ctsy

set ip next-hop recursive X.X.83.248   <•• Route is not seen anymore

set default interface GigabitEthernet0/0.100 <•• And it doesn’t care about this statement since the default route has a higher priority… <•• (my current conclusion..?)

At this moment I am trying to find a solution for the current situation, which is that the customer is sending a default route and I have to somehow bypass this route in order to get offload work seamlessly again!

As soon as I have found any results I will post it and if you have any suggestions please let me know.

787
Views
0
Helpful
19
Replies