Usage of Route Maps for Next Hop

Answered Question
Apr 18th, 2012

Hi Everyone,

I have a Cisco L3 switch that I have configured route maps on to amend the next hop to be a firewall. The destination network for the traffic is also connected to the switch (therefore directly connected network), but my issue is this.

If the FW fails, then the traffic will still try to be sent to the down FW due to the route map amending the next hop. Is there a way that I can get the traffic to go via the connected network if the FW should fail? As far as I am aware, the route map will amend the next hop to the FW IP whether the FW is up or not, and therefore the traffic will be dropped.

Am I right on this or has anyone got another idea?

Thanks in advance,

Dan

I have this problem too.
0 votes
Correct Answer by dancicioiu about 2 years 17 hours ago

Could you tell me the source and the destination of the packet ?

Dan

  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 5 (1 ratings)
John Blakley Wed, 04/18/2012 - 12:17

Daniel,

I haven't labbed this up, but here's what the docs say:

The set ip next-hop command verifies the existence of the next hop specified, and…

  • if the next hop exists in the routing table, then the command policy routes the packet to the next hop.

  • if the next hop does not exist in the routing table, the command uses the normal routing table to forward the packet.

OR

set ip next-hop verify-availability

To configure policy routing to verify the reachability of the next hop  of a route map before the router performs policy routing to that next  hop, use the set ip next-hop verify-availability command in route-map configuration mode. To disable this function, use the no form of this command.

set ip next-hop verify-availability [next-hop-address sequence track object]

no set ip next-hop verify-availability [next-hop-address sequence track object]

If your IOS supports it, you could see if the verify-availability is available. Otherwise, how are you routing to the firewall? Can you post a quick diagram of how this is laid out?

John

danbowencisco Wed, 04/18/2012 - 13:22

thanks John but I did some research and this method uses CDp to discover if the next hop is available or not. Unfortunately, my next hop is an ASA and therefore doesnt run CDP .

Thanks anyway,


Dan

dancicioiu Wed, 04/18/2012 - 13:26

Hi Dan,

As far as I know - and also from John's post - the set ip next-hop verify has a tracking option

Dan

Richard Burts Wed, 04/18/2012 - 13:27

Dan

It is one factor about setting a static route next hop or a Policy Based Routing next hop when the next hop is on an Ethernet interface that IOS will use the next hop as long as the router interface on that subnet is in the up state. So the next hop device (firewall in your case) could lose connectivity but the router will continue to try to use the next hop resulting in dropping traffic.

The alternative identified by John to use verify-availability in PBR is quite effective. I have implemented it a couple of times and it works quite well. I suggest that you give this a try.

HTH

Rick

[edit] I am not sure where Dan did his research, but I have done the verify availability using ping to the next hop and not using CDP. I do not know if it is a specific version dependent thing. But I know that at least for some versions of IOS the verify-availability of PBR can use ping.

danbowencisco Wed, 04/18/2012 - 13:40

can you tell me how you did this Rick?

I am using a 3560G and the verify availability command does not give me the option to set the method of verifying.

Thanks,


Dan

dancicioiu Wed, 04/18/2012 - 13:45

Hi,

If there are no odd requirements - only the traffic sourced from subnet x should go to the firewall  - also the static route is an option, together with tracking option ( sla icmp ) which will solve both link or any other issues.

As Rick wrote , you will route as long as the interface will be up,  but nowadays almost all firewalls are in cluster and connected to different access switches making this hard - even by loosing the link to the firewall - as a SVI to go down ( because of the link (trunk) between the switches ).

He is right Rick :

The set ip next-hop verify-availability command can be used in the following two ways:

With policy-based routing (PBR) to verify next hop reachability using Cisco Discovery Protocol (CDP).

http://www.cisco.com/en/US/docs/ios/12_3t/12_3t4/feature/guide/gtpbrtrk.html

Dan

danbowencisco Wed, 04/18/2012 - 13:44

the ip sla monitor command is not an option on a 3560G (at least not on 12.2.58 SE2).

This is what I get...

set ip next-hop verify-availability 10.11.120.161 1 track 1

for ip sla I only get these options...no monitor

SL-Cisco-3560G-SW(config)#ip sla ?

  <1-2147483647>          Entry Number

  enable                  Enable Event Notifications

  group                   Group Configuration or Group Scheduling

  key-chain               Use MD5 Authentication for IP SLAs Control Messages

  logging                 Enable Syslog

  low-memory              Configure Low Water Memory Mark

  reaction-configuration  IP SLAs Reaction-Configuration

  reaction-trigger        IP SLAs Trigger Assignment

  read                    Read data for use with IP SLA

  reset                   IP SLAs Reset

  responder               Enable IP SLAs Responder

  restart                 Restart An Active Entry

  schedule                Entry Scheduling

danbowencisco Wed, 04/18/2012 - 13:49

ah brilliant, thanks Dan .

Now I need to go about setting it to use ICMP - not sure how to do this yet.

Dan

dancicioiu Wed, 04/18/2012 - 13:56

ip sla 1

icmp-echo 10.11.120.161

timeout 300

threshold 300

freq 2

ip sla sched 1 lif fo start no

track 1 ip sla 1

show track

Dan

dancicioiu Wed, 04/18/2012 - 13:59

I think that we have different IOS versions :

ip sla monitor 1

type echo protocol ipicmpecho 10.11.120.161

ip sla sched 1 lif fo start no

track 1 ip sla 1

show track

Dan

danbowencisco Wed, 04/18/2012 - 13:47

my route map says this....

route-map NMS-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2550

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [undefined]

    ip next-hop 10.11.120.161

danbowencisco Wed, 04/18/2012 - 14:07

Hi Dan,

Did what you suggested in the first reply and I get the following:

Not sure why it says tracked by Route Map 0 - not sure where I have the option to point it at NMS-RM route map.

SL-Cisco-3560G-SW#sh track

Track 1

  IP SLA 1 state

  State is Down

    1 change, last change 00:00:04

  Latest operation return code: Unknown

  Tracked by:

    ROUTE-MAP 0

SL-Cisco-3560G-SW#sh route-map NMS-RM

route-map NMS-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2550

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [down]

    ip next-hop 10.11.120.161

  Policy routing matches: 18619 packets, 1578776 bytes

dancicioiu Wed, 04/18/2012 - 14:11

I belive from "Latest operation return code: Unknown" that you have to also set the frequency.

Please paste "show ip sla configuration"

Dan

danbowencisco Wed, 04/18/2012 - 14:13

SL-Cisco-3560G-SW#sh ip sla configuration

IP SLAs Infrastructure Engine-III

Entry number: 1

Owner:

Tag:

Operation timeout (milliseconds): 5000

Type of operation to perform: icmp-echo

Target address/Source address: 10.11.120.161/0.0.0.0

Type Of Service parameter: 0x0

Request size (ARR data portion): 28

Verify data: No

Vrf Name:

Schedule:

   Operation frequency (seconds): 60  (not considered if randomly scheduled)

   Next Scheduled Start Time: Pending trigger

   Group Scheduled : FALSE

   Randomly Scheduled : FALSE

   Life (seconds): 3600

   Entry Ageout (seconds): never

   Recurring (Starting Everyday): FALSE

   Status of entry (SNMP RowStatus): notInService

Threshold (milliseconds): 5000

Distribution Statistics:

   Number of statistic hours kept: 2

   Number of statistic distribution buckets kept: 1

   Statistic distribution interval (milliseconds): 20

Enhanced History:

History Statistics:

   Number of history Lives kept: 0

   Number of history Buckets kept: 15

   History Filter Type: None

dancicioiu Wed, 04/18/2012 - 14:17

The funny thing is that in order to modify it we should stop it :

no ip sla sched 1 lif fo start no

ip sla monit 1

timeout 300

threshold 300

freq 2

ip sla sched 1 lif fo start no

Dan

danbowencisco Wed, 04/18/2012 - 14:17

SL-Cisco-3560G-SW#sh track

Track 1

  IP SLA 1 state

  State is Down

    1 change, last change 00:11:26

  Latest operation return code: Unknown

  Tracked by:

    ROUTE-MAP 0

SL-Cisco-3560G-SW#sh route-map NMS-RM

route-map NMS-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2550

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [down]

    ip next-hop 10.11.120.161

  Policy routing matches: 18788 packets, 1592603 bytes

danbowencisco Wed, 04/18/2012 - 14:19

BRILLIANT!!!

we are good. Thanks so much Dan.

SL-Cisco-3560G-SW#sh track

Track 1

  IP SLA 1 state

  State is Up

    2 changes, last change 00:00:31

  Latest operation return code: OK

  Latest RTT (millisecs) 1

  Tracked by:

    ROUTE-MAP 0

SL-Cisco-3560G-SW#sh rou

SL-Cisco-3560G-SW#sh route-map NMS-RM

route-map NMS-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2550

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [up]

    ip next-hop 10.11.120.161

  Policy routing matches: 18808 packets, 1594213 bytes

danbowencisco Wed, 04/18/2012 - 14:39

well, the tracking appears to be working but its not doing what I expect.

When I take the FW interface down, the traffic drops and doesnt route via the switch - not sure why.

Dan

dancicioiu Wed, 04/18/2012 - 14:44

Take a look :

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [up]

    ip next-hop 10.11.120.161

Dan

Richard Burts Wed, 04/18/2012 - 15:16

Dan

Would you post the configuration of the route map? And perhaps post the output of show route-map taken at a time when the firewall interface is down?

HTH

Rick

danbowencisco Thu, 04/19/2012 - 01:18

Ok, here is a drawing of what I am working with.

What I am trying to achieve is the ability to route via the switch (directly connected) should the FW interface or Firewall fail.

Dan, here is the route map output when both G0.0/128 and G0/0.160 FW interfaces are down.

SL-Cisco-3560G-SW#sh route-map NMS-RM

route-map NMS-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2550

  Set clauses:

    ip next-hop verify-availability 10.11.120.161 1 track 1  [down]

    ip next-hop 10.11.120.161

  Policy routing matches: 26501 packets, 2254398 bytes

SL-Cisco-3560G-SW#sh route-map Supervisory-RM

route-map Supervisory-RM, permit, sequence 10

  Match clauses:

    ip address (access-lists): 2540

  Set clauses:

    ip next-hop verify-availability 10.11.120.129 1 track 2  [down]

    ip next-hop 10.11.120.129

  Policy routing matches: 1276 packets, 113744 bytes

This is what I would expect as the next hop is now unreachable and therefore I would expect the traffic to be forwarded by the switch using the connected destination network, shown in the routing table from the switch. Problem is, the traffic times out when pinging between hosts 10.11.120.163 to 10.11.120.131.

SL-Cisco-3560G-SW#sh ip route 10.11.120.131

Routing entry for 10.11.120.128/28

  Known via "connected", distance 0, metric 0 (connected, via interface)

  Routing Descriptor Blocks:

  * directly connected, via Vlan128

      Route metric is 0, traffic share count is 1

SL-Cisco-3560G-SW#sh ip route 10.11.120.163

Routing entry for 10.11.120.160/28

  Known via "connected", distance 0, metric 0 (connected, via interface)

  Routing Descriptor Blocks:

  * directly connected, via Vlan160

      Route metric is 0, traffic share count is 1

route-map NMS-RM permit 10

match ip address 2550

set ip next-hop verify-availability 10.11.120.161 1 track 1

set ip next-hop 10.11.120.161

!

route-map Supervisory-RM permit 10

match ip address 2540

set ip next-hop verify-availability 10.11.120.129 1 track 2

set ip next-hop 10.11.120.129

access-list 2540 remark ***Supervisory Network Route Map ACL***

access-list 2540 deny   icmp any host 10.11.120.130

access-list 2540 permit ip 10.11.120.128 0.0.0.15 any

access-list 2550 remark ***Network Managment Network Route Map ACL***

access-list 2550 deny   icmp any host 10.11.120.162

access-list 2550 deny   udp host 10.11.120.163 host 10.11.120.162 eq snmp

access-list 2550 deny   udp host 10.11.120.163 host 10.11.120.162 eq snmptrap

access-list 2550 permit ip 10.11.120.160 0.0.0.15 any

I took the route maps from the interface and it starts to work, however, it will now not let me put them back. It accepts the command but the RM config does not show in a show int vlan 128.

Very strange.

dancicioiu Thu, 04/19/2012 - 01:21

Remove:

set ip next-hop 10.11.120.161

Your route-map should look like that :

route-map NMS-RM permit 10

match ip address 2550

set ip next-hop verify-availability 10.11.120.161 1 track 1

Dan

danbowencisco Thu, 04/19/2012 - 01:23

ok Dan, thank you.

As soon as the switch lets me put the RM statement back onto the interface I will (reloading).

Dan

danbowencisco Thu, 04/19/2012 - 01:39

OK my config is now this, reapplied it after reload (I standardised the numbering to match the IP addressing).

Still timing out when I shut down both FW interfaces.

ip sla 160
icmp-echo 10.11.120.161 source-ip 10.11.120.162
threshold 300
timeout 300
frequency 8
ip sla schedule 160 life forever start-time now

!
track 160 ip sla 160


route-map NMS-RM permit 10
match ip address 2550
set ip next-hop verify-availability 10.11.120.161 1 track 160

ip sla 128
icmp-echo 10.11.120.129 source-ip 10.11.120.130
threshold 300
timeout 300
frequency 8
ip sla schedule 128 life forever start-time now

track 128 ip sla 128

route-map Supervisory-RM permit 10
match ip address 2540
set ip next-hop verify-availability 10.11.120.129 1 track 128

dancicioiu Thu, 04/19/2012 - 01:48

Now , in my opinion , the PBR works as expected.

If the next-hop is reachable the traffic is forwarded to the firewall.

If the next-hop is not reachable the traffic is forworded according to the routing table.

Where does the routing table route the traffic ?

show ip route

Dan

danbowencisco Thu, 04/19/2012 - 01:53

It should route it straight out of the VLAN 128 interface which is directly connected to the switch.

The destination device is physically connected to the switch on VLAN 128, so I cant see why its not working. If I remove the route map statement from the interface, it works.

SL-Cisco-3560G-SW#sh ip route 10.11.120.131

Routing entry for 10.11.120.128/28

  Known via "connected", distance 0, metric 0 (connected, via interface)

  Routing Descriptor Blocks:

  * directly connected, via Vlan128

      Route metric is 0, traffic share count is 1

Correct Answer
dancicioiu Thu, 04/19/2012 - 01:57

Could you tell me the source and the destination of the packet ?

Dan

dancicioiu Thu, 04/19/2012 - 02:15

You have also a PBR there : Supervisory-RM -> next-hop 10.11.120.129.

Beyond the initial query , what are you trying to achieve ?

What does the access-lists 2540 and 2550 contain ?

Dan

danbowencisco Thu, 04/19/2012 - 02:21

I am trying to configure the routing so that if they FW is down, the traffic will be routed via the switch. In normal operation, the packet hits the gateway (VLAN 160), the route map is applied and the next hop is set to the FW interface (120.161). Should the FW fail, the verify availability command should pick this up via icmp and not use the route map, instead using the routing table where it see's the destination address as being directly connected. Trouble is, unless I remove the route map, it times out.

The PBR isnt being used though as I have used the verify availability command on Supervisory-RM and shut down the FW interface for that VLAN.

The access lists are purely for the route map traffic, they are below:

SL-Cisco-3560G-SW#sh access-list 2550

Extended IP access list 2550

    10 deny icmp any host 10.11.120.162

    20 deny udp host 10.11.120.163 host 10.11.120.162 eq snmp

    30 deny udp host 10.11.120.163 host 10.11.120.162 eq snmptrap

    40 permit ip 10.11.120.160 0.0.0.15 any (7546 matches)

SL-Cisco-3560G-SW#sh access-list 2540

Extended IP access list 2540

    10 deny icmp any host 10.11.120.130

    20 permit ip 10.11.120.128 0.0.0.15 any (1077 matches)

danbowencisco Thu, 04/19/2012 - 02:42

my mistake Dan, you were right. I hadnt taken off the set ip next hop 10.11.120.129 - it works now!

Thank you so much!!!!

Dab

dancicioiu Thu, 04/19/2012 - 02:43

Daniel ,

You have some issues in your setup :

     1) what if just one interface of the firewall will fail ? =>just one track will fail , and all the traffic will be drop. in order to solve this issue you should use on the next-hop check availability a combined track of the onces already configured

   track 66 list boolean or

          object 128

          object 160

In case of any of the two track will fail , this track will fail. This track id should be used on both route-maps.

    2) in Failed mode the traffic is by-passed from the firewall, in case of recovery all the current traffic flows will be droped by the firewall.

[ Later edit ]

I have a strange fealing about this

I re-read your posts and : Do you get any error messages when you apply the route-map int the interface vlan 128 ?

Could you post "show ip policy"

Dan

danbowencisco Thu, 04/19/2012 - 02:49

thanks for the info regarding the tracking, I will apply that now.

Regarding failed mode, you mean when the FW recovers it will still route via the switch?

Dan

danbowencisco Thu, 04/19/2012 - 02:53

once the FW is back online, shouldnt the verify availability pick up that the FW is reachable and start using the route map again?

Dan

I have noticed when I shut down the FW ints, I get a 6 or 7 sec delay before traffic is rerouted, when I no shut the ints, it doesnt drop at all (as if it isnt failing back).

dancicioiu Thu, 04/19/2012 - 03:00

Daniel,

The switch will check every 2 seconds if the FW is up or not. This is not the problem.

From the moment of firewall failure, when the traffic si routed directly to the moment when the firewall will recover, let's consider some active flows. When the firewall will recover the flows will be droped, because there is no info about those flows. Those flows must re-initalise in order to work, and I am not talking about icmp, but about tcp/udp flow. This is the way that a statefull firewall would work

Dan

danbowencisco Thu, 04/19/2012 - 03:56

ah right, I understand. So the connection will need to be torn down and re-established for the traffic to pass. I understand now. This may not be too much of an issue.

Thank you so much for your time and help.


Dan

danbowencisco Thu, 04/19/2012 - 04:23

reason I ask is I have 6 VLANs in total. If VLAN 1 goes down I do not want traffic from VLAN 160 to 128 being routed via the switch as those interfaces are still up, I only want VLAN 1 to not have its next hop amended.

Is this possible?

Also, if I use the track list command, we are no longer getting it to track an IP SLA and therefore it will not be using icmp to monitor the next hop?

dancicioiu Thu, 04/19/2012 - 05:07

Dan,

Could you be more clear with what you what to achieve ?

Currently you have vlan 160 and 128 , and all traffic source from thouse vlans is forwarded to the Firewall based on the access-list applied on the route-map. This affectes only traffic sourced by this vlans. If you have other 4 vlans , the returning traffic will bypass the FW. The result will be Vlan 160, and 128 cannot communicate with the remaining vlans.

Dan

danbowencisco Thu, 04/19/2012 - 05:25

Sorry Dan.

In total I have VLANs 16,32,48,96,128 and 160. All VLANs have a route map applied amending their next hop to be their FW interface.

Should the FW fail completely, I want all traffic to be routed internally to the switch and not have the route map amend the next hop to the FW.

Should a particular interface on the FW fail, for example, VLAN 128, then I want traffic between VLAN 128 and all other VLANs to go via the switch. I do not want traffic between say VLANs 96 and 160 to be routed via the switch as they are not affected by the FW interface going down.

My opinion is that if I configure it as we discussed, should any VLAN interface go down, everything using the object tracking list (all VLANs) will route via the switch.

I want to avoid this if possible.

Thank you,


Dan

dancicioiu Thu, 04/19/2012 - 05:46

Dan ,

I think that it is posible , but there is long way to configure it :

  - for each vlan you have a PBR , the route-map should have 5 entries, for each other vlan.

  - you should track each FW IP

  - track combined between very vlan

Here is for example the config only for VLAN128.

ip access-l ex VLAN128-VLAN16

permit ip VLAN128 VLAN16

ip access-l ex VLAN128-VLAN32

permit ip VLAN128 VLAN32

ip access-l ex VLAN128-VLAN48

permit ip VLAN128 VLAN48

ip access-l ex VLAN128-VLAN96

permit ip VLAN128 VLAN96

ip access-l ex VLAN128-VLAN160

permit ip VLAN128 VLAN160

==== TRACK each VLAN

track 1

ping FW-VLAN128

track 2

ping FW-VLAN16

track 3

ping FW-VLAN32

track 4

ping FW-VLAN48

track 5

ping FW-VLAN96

track 6

ping FW-VLAN160

===== TRACK combined

track 10 list boo or

track 1

track 2

track 20 list boo or

track 1

track 3

track 30 list boo or

track 1

track 4

track 40 list boo or

track 1

track 4

track 50 list boo or

track 1

track 5

track 60 list boo or

track 1

track 6

route-map PBR-VLAN128 permit 20

match ip address VLAN128-VLAN16

set ip next-hop x.x.x.x verify-reach track 20

route-map PBR-VLAN128 permit 30

match ip address VLAN128-VLAN32

set ip next-hop x.x.x.x verify-reach track 30

route-map PBR-VLAN128 permit 40

match ip address VLAN128-VLAN48

set ip next-hop x.x.x.x verify-reach track 40

route-map PBR-VLAN128 permit 50

match ip address VLAN128-VLAN96

set ip next-hop x.x.x.x verify-reach track 50

route-map PBR-VLAN128 permit 60

match ip address VLAN128-VLAN160

set ip next-hop x.x.x.x verify-reach track 60

Dan

danbowencisco Thu, 04/19/2012 - 06:00

thats brilliant Dan.

I have just one question...

On the "track each VLAN" section, I apply a track ip sla (number) here?

then, within the SLA, I use icmp echo and ping between VLAN 128 switch to the FW IP of the VLAN in question? Like this...

example between 128 and 160

ip sla 1

icmp-echo 10.11.120.161 source ip 10.11.120.130

threshold 300

blah

blah

is that what you meant?


Dan

dancicioiu Thu, 04/19/2012 - 06:05

Not quite. First there is no need for the source IP , it wil use the IP on the interface vlan.

ip sla 1

icmp-echo FW-VLAN128

freq ...

threshold ...

timeout ...

ip sla 1 schedule ...

track 1 ip sla 1

ip sla 2

icmp-echo FW-VLAN32

freq ...

threshold ...

timeout ...

ip sla 2 schedule

track 2 ip sla 2

....so on

Dan

danbowencisco Thu, 04/19/2012 - 06:09

fantastic, thanks so much Dan, Ive learned a lot and you have really helped.

All the best and thanks again.

Dan

danbowencisco Thu, 04/19/2012 - 10:45

All of the config was good, it just wont allow me to place the route map on the interface. it doesnt error, just isnt in the config when I apply it.

It will allow me to add an older RM to the interface, just not one of these new ones.

I really dont know why.

Dan

Actions

Login or Register to take actions

This Discussion

Posted April 18, 2012 at 9:02 AM
Stats:
Replies:70 Avg. Rating:5
Views:2757 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard

Rank Username Points
1 15,007
2 8,150
3 7,730
4 7,083
5 6,742
Rank Username Points
160
77
70
69
50