05-20-2009 12:58 AM - edited 03-04-2019 04:49 AM
Hi!
We have 2 internet gateways (ISP-A and ISP-B), so I have made a configuration using tracks to set Gateway of last resort depending on what ISP is up.
This seems to work perfectly fine.
BUT; In addition to this, only a few of our subnets are "allowed" to use ISP-B, so I have made a route-map:
route-map test permit 10
match ip address 10
set ip next-hop verify-availability 158.38.16.129 100 track 1
set ip next-hop verify-availability 213.184.198.1 101 track 2
And this works, even though 213.184.198.1 is the default gateway, the traffic is re-routed because of this route-map.
The problem starts when 158.38.16.129 dissapeares. The other gateway, 213.184.198.1 should then take over, but it does not. Even though the trackers clearly says that the gateway is down:
InternettGW#sh track
Track 1
Response Time Reporter 1 reachability
Reachability is Down
5 changes, last change 00:29:57
Delay down 5 secs
Latest operation return code: Timeout
Tracked by:
ROUTE-MAP 0
STATIC-IP-ROUTING 0
Track 2
Response Time Reporter 2 reachability
Reachability is Up
3 changes, last change 00:37:47
Delay down 5 secs
Latest operation return code: OK
Latest RTT (millisecs) 1
Tracked by:
ROUTE-MAP 0
STATIC-IP-ROUTING 0
InternettGW#sh route-map
route-map test, permit, sequence 10
Match clauses:
ip address (access-lists): 10
Set clauses:
ip next-hop verify-availability 158.38.16.129 100 track 1 [down]
ip next-hop verify-availability 213.184.198.1 101 track 2 [up]
Policy routing matches: 30 packets, 3500 bytes
Any ideas why?
I am posting the complete configuration if someone is interested.
Thank you.
----------------------------------------------------------------------------
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname InternettGW
!
boot-start-marker
boot system flash:c1841-advipservicesk9-mz.124-4.XC7.bin
boot-end-marker
!
enable secret 5 $1$EBrC$vpM9NVPlgxLd2SpRe0ymi0
enable password whatteevver
!
no aaa new-model
!
resource policy
!
ip cef
!
!
!
!
ip name-server 213.184.198.11
ip sla 1
icmp-echo 158.38.0.168 source-interface FastEthernet0/0
frequency 5
ip sla schedule 1 life forever start-time now
ip sla 2
icmp-echo 213.184.200.1 source-interface FastEthernet0/1
frequency 5
ip sla schedule 2 life forever start-time now
!
!
!
archive
log config
hidekeys
!
!
track 1 rtr 1 reachability
delay down 5
!
track 2 rtr 2 reachability
delay down 5
!
!
!
!
!
interface FastEthernet0/0
description * ISP-B *
ip address 158.38.16.140 255.255.255.128
ip policy route-map test
duplex auto
speed auto
!
interface FastEthernet0/1
description * ISP-A *
ip address 213.184.198.137 255.255.255.0
ip policy route-map test
duplex auto
speed auto
!
interface FastEthernet0/0/0
switchport mode trunk
!
interface FastEthernet0/0/1
shutdown
!
interface FastEthernet0/0/2
shutdown
!
interface FastEthernet0/0/3
shutdown
!
interface Vlan1
no ip address
shutdown
no mop enabled
!
interface Vlan53
ip address 10.50.53.10 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 158.38.16.129 101 track 1
ip route 0.0.0.0 0.0.0.0 213.184.198.1 100 track 2
ip route 10.10.10.205 255.255.255.255 10.50.53.1
ip route 10.50.50.240 255.255.255.255 10.50.53.1
ip route 158.38.0.168 255.255.255.255 158.38.16.129
ip route 213.184.200.1 255.255.255.255 213.184.198.1
!
!
ip http server
no ip http secure-server
!
access-list 10 permit 158.38.16.142 // Thisis the ip address I'm testing with.
!
!
!
route-map test permit 10
match ip address 10
set ip next-hop verify-availability 158.38.16.129 100 track 1
set ip next-hop verify-availability 213.184.198.1 101 track 2
!
!
!
!
control-plane
!
!
!
line con 0
line aux 0
line vty 0 4
password whatevvere
login
!
scheduler allocate 20000 1000
!
webvpn context Default_context
ssl authenticate verify all
!
no inservice
!
end
05-20-2009 06:07 AM
Hi,
From what I see, your source is on the same subnet as your router and the ISP gateway.
Could you check if your router is sending ICMP redirect messages with a debug icmp command. If it's the case, your host may cache this information for a while...
you can disable ICMP redirect messages generation with the no ip redirect interface command.
More info: http://www.cisco.com/en/US/tech/tk365/technologies_tech_note09186a0080094702.shtml
HTH
Laurent.
05-20-2009 09:09 AM
Thank you for your reply.
Yes, the source will always be on the same subnet as my router and ISP Gateway. Is it better if it comes from a third subnet?
when turning on debug, and try a traceroute, I get these messages:
*May 20 16:58:54.318: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
*May 20 16:58:54.318: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
*May 20 16:58:54.318: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
Nothing else is seen, except the same info over and again from the 2 trackers:
*May 20 17:19:13.123: ICMP: echo reply rcvd, src 213.184.200.1, dst 213.184.198.137
*May 20 17:19:13.127: ICMP: echo reply rcvd, src 158.38.0.168, dst 158.38.16.140
(The last one is not seen of course, when I shut down that gateway)
It seems that something is looping somewhere, but why?
Thank you.
05-20-2009 10:36 AM
I ran into the same problem. The next-hop in the route-map never worked for me either.
I eventually had to do the following:
ip sla monitor 1
type echo protocol ipIcmpEcho XXX.XXX.XXX.XXX source-interface GigabitEthernet0/1
frequency 15
ip sla monitor schedule 1 life forever start-time now
ip sla monitor 2
type echo protocol ipIcmpEcho 192.168.28.3 source-interface Serial1/0
frequency 15
ip sla monitor schedule 2 life forever start-time now
track 101 rtr 1 reachability
delay down 60 up 10
!
track 102 rtr 2 reachability
interface GigabitEthernet0/0
ip address 10.0.0.254 255.255.255.0
no ip redirects
no ip unreachables
ip flow ingress
ip policy route-map SETGATEWAY
duplex full
speed 1000
media-type rj45
ip route 0.0.0.0 0.0.0.0 192.168.29.3 track 101
ip route 0.0.0.0 0.0.0.0 192.168.30.1 254 track 102
access-list 2001 permit ip 192.168.0.0 0.0.255.255 any log-input
access-list 2001 permit ip 172.16.0.0 0.0.255.255 any log-input
route-map SETGATEWAY permit 10
match ip address 2001
set ip default next-hop 192.168.29.3
05-21-2009 11:57 PM
bryantmarsh, thank you for your answer.
I have tried your config now, but it still don't work :(
05-20-2009 05:07 PM
Hi,
What you see is expected, Traceroute increment the TTL starting with one up to the destination is reach to discover each host on the path.
You don't have to do anything except waiting to see if ICMP redirect are generated by the router.
The cache timeout due to ICMP redirect could be long (10mn by default on windows server). So if it's the root cause, you should see an ICMP redirect from the debug icmp each time the host cache timeout.
You should check your host to see if there is any default route pointing directly to 158.38.16.129 or disable ICMP redirect on the router and see if it resolves your issue.
From a design perspective, I agree that this host should be on another link. In this case apply the route-map on this new interface. It's safer to have your public host behind a router so you can better protect it.
Actually, you don't need to specify the 2nd ip next-hop in your route-map. If 158.38.16.129 is unavailable, the routing table will be used and we know the installed default route already points to the other ISP.
HTH
Laurent.
05-22-2009 12:08 AM
Hello again.
I cannot see any ICMP Redirects in the debug, no matter how long it stays on.
I really don't think this has something do to with the host itself.
There are noe default routes pointing to 158.38.16.129 on the host either.
I have also tried to disable ICMP redirects, but no difference :(
I'm running out of options here :(
About the design;
The intention was to put this router in front of our existing firewall, so we could use the existing rules from it.
And the traffic coming through the Firewall is already NAT'ed.
The plan was to use 2 NAT ip addresses from the firewall to the router to seperate the traffic coming from the inside:
Example:
Traffic from 10.20.x.x will be seen as 213.184.198.150 on the outside.
Traffic from 10.30.x.x will be seen as 158.38.16.150 on the outside.
Since the firewall is not capable of PBR, I was hoping to do it with my Cisco Router, as explained above.
We are really close now, but no cigar :)
One thing, though:
Even when both ISP's are up, and the traffic is flowing as intended, I still get this messages from the debug ip icmp:
*May 22 08:09:13.622: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
*May 22 08:09:13.622: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
*May 22 08:09:13.622: ICMP: time exceeded (time to live) sent to 158.38.16.142 (dest was 195.139.129.130)
Isn't that strange?
Thanks.
05-22-2009 05:08 AM
Hello again.
A little update on this:
It seems that you were right.
After some more testing, it looks that the failover works perfectly fine, but ONLY on ip addresses it does not know about from before.
IP Adresses it DOES know about, starts working again after a time (30 mins?)
So obviously there is a cache somewhere. But where? Not on the remote servers or anywhere on the remote side I hope, then we have a big problem :)
And how to bypass it?
Thank you.
05-22-2009 06:57 AM
Hi,
You need to check the documentation of the OS of your server to see how it handles ICMP redirect messages.
HTH
Laurent.
05-25-2009 03:50 AM
Hello again,
It is working now.. sometimes. And sometimes it don't
I don't really get it.
But you are saying that I need to disable ICMP redirect on whatever host I choose to test this with?
And I don't have to disable ICMP redirects on the router itself where this configuration is taken from?
So if I choose to test this with another Cisco router (or L3 Switch), this should be no problem?
Thanks again,
- Ãystein
05-25-2009 05:40 AM
Hi,
Only routers generate ICMP redirect messages and Host must know how to process them (part of the ICMP stack).
In your case, disable ICMP redirect on the router interface pointing to your host. Wait enough time to be sure your host cache information timeout (or clear it if you know how)and repeat your test. Your host should be able to reach any Internet sites.
HTH
Laurent.
05-26-2009 05:06 AM
Hi.
Traffic from the interface where I have to use route-map to route it through the right interface works very well now.
But traffic from the interface that has the default route originally will not fail over when the default route changes.
I use another Cisco 1841 to test this with now, and here are the two configurations:
The router I'm testing from:
-------
!
ip name-server 213.184.198.11
ip name-server 213.184.200.1
ip name-server 158.38.0.168
!
interface FastEthernet0/0
ip address 158.38.16.150 255.255.255.128
no ip redirects
no ip route-cache
duplex auto
speed auto
no mop enabled
!
interface FastEthernet0/1
ip address 213.184.198.88 255.255.255.0
no ip redirects
no ip route-cache
duplex auto
speed auto
!
ip default-gateway 213.184.198.137
----------------------------
This is the Internet Gateway as it stands right now:
-----------------------------
hostname InternettGW
ip name-server 213.184.198.11
ip name-server 158.38.0.168
ip sla 1
icmp-echo 158.38.0.168 source-interface FastEthernet0/0
frequency 5
ip sla schedule 1 life forever start-time now
ip sla 2
icmp-echo 213.184.200.1 source-interface FastEthernet0/1
frequency 5
ip sla schedule 2 life forever start-time now
!
!
track 1 rtr 1 reachability
delay down 5 up 5
!
track 2 rtr 2 reachability
delay down 5 up 5
!
interface FastEthernet0/0
ip address 158.38.16.143 255.255.255.128
no ip redirects
duplex auto
speed auto
!
interface FastEthernet0/1
ip address 213.184.198.137 255.255.255.0
no ip redirects
ip policy route-map test
duplex auto
speed auto
!
interface FastEthernet0/0/0
switchport mode trunk
!
interface Vlan53
description * Administration Interface Only! *
ip address 10.50.53.10 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 158.38.16.129 101 track 1
ip route 0.0.0.0 0.0.0.0 213.184.198.1 100 track 2
ip route 10.0.0.0 255.0.0.0 10.50.53.1
ip route 158.38.0.168 255.255.255.255 158.38.16.129
ip route 213.184.200.1 255.255.255.255 213.184.198.1
!
!
!
access-list 10 permit 158.38.16.142
access-list 10 permit 158.38.16.150
!
!
!
route-map test permit 10
match ip address 10
set ip next-hop verify-availability 158.38.16.129 10 track 1
!
!
----------------------------------
Any tips on how to complete this final stage?
Thanks again,
- Ãystein
05-26-2009 05:21 AM
The route-map shouldn't be on F0/0 instead of 0/1 ?
Laurent.
05-26-2009 05:46 AM
No I think not, since the default gateway on the test router is 213.184.198.137.
Therefore, it is F0/1 that will recieve all packets.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide