Redundant/Load balance with 2 ISPs

[corvallisclinic]

I have a need to implement redundancy with load balancing (outbound) to two separate ISPs using two separate routers (hardware redundancy). One ISP is over T1 the other is cable. The network basically has core routers connected to two WAN routers, one for each of the ISPs. Traffic from the core should be load balanced over the WAN routers to the separate ISPs and if one connection fails, all traffic diverted to the other.

Both WAN routers are inside separate firewalls. The T1 line connects to a T1 router then the firewall.

I considered having the firewalls advertise the default routes to the WAN routers then injecting that route into the core, but how would it learn the down state if the cable connection fails? This also could pose a security issue (RIP outside the firewall).

BGP is not an option as it is not offered by the cable provider.

I also wondered about Policy Based Routing to determine the next hop for the cable connection.

Any help would be much appreciated!

Network as follows:

---------WAN1-- FW1-- Cable Modem---ISP1

Core--/

\---------WAN2--FW2--T1 Router---ISP2

[Georg Pauwen]

Hello,

the easiest way to accomplish load balancing is to turn on CEF, and then configure two equal cost static routes. Let´s say on your core router you are using Serial0 and Serial1 for your outbound connections, your configuration would look like this:

ip cef

!

ip route 0.0.0.0 0.0.0.0 Serial0

ip route 0.0.0.0 0.0.0.0 Serial1

CEF will install both routes and load balance over both of them...

Regards,

GP

[dknov]

Hi,

Well, CEF is great, but not for this ocasion :-)

What you need is a conditional default-route advertisement and you don't need a BGP peering with your ISP for this. There might be a few ways to do it, but here is one.

Create two IBGP sessions one from Cable router and the other from T1 router to a device(s) in your LAN.

Per-each of those sessions configure BGP to source default-route while using route-map option. Inside the route-map do the matching for a connected interface on each router respectively (Cable interface and Serial interface).

What you get is that once one of the links goes down, respective interface will go down and route-map condition will not be satisfied, which will in turn stop the router from advertising default-route.

One thing you of course need to take care is redistribute IBGP into your IGP inside the LAN.

Now, how do you make this setup woth in active/backup mode, meaning one link is always used (I assume the T1) while the other says for backup?

Well it depends on your IGP. RIP, for example, has an option of setting the metric to 'transparent' at the redistribution point, so you can tweak BGPs MED value to be used in RIP domain for metric....But if RIP is not an IGP for your choise, you need to see how you can do it with your IGP.

Hope that helped.

David

[mheusinger]

Hi David,

sorry to disagree. Your solution might work, but is too complex for the scenario given. Basically the two static default routes will behave like your next hop in the route-map. Once the interface is down they will be withdrawn. And the request was to get load sharing over both access lines as long as they are up - something your solution might not deliver.

Regards,

Martin

So no gain I can see. Take the two defaults, redistribute them into IGP and that´s it.

Hope this helps

Martin

[corvallisclinic]

That was kind of what I was thinking, but to include redundancy I was looking at placing the static default routes in the WAN routers then using PBR to point to the next hop outside the network, one on each WAN router to its specific Internet access. If it can not reach the next hop to then point to the other WAN router. Load balancing would happen with maybe EIGRP and varience to the WAN routers from the core routers after redistributing the static routes on the WAN routers into EIGRP.

Is that a possiblity?

I agree to keep it as simple as possible.

Thanks all for the ideas!

Erik

[dknov]

Hi,

I agree, the simpler the better, however I am not sure that this simplicity will work in this case :-)

First, about using PBR. Do you mean using "set ip default next-hop" in a route-map? Route-maps do not have ability to perform "next hop tracking", so if you configure this route-map on your WAN routers, your WAN routers will keep forwarding traffic in a direction of failed link.

Now about reditributing statics into IGP. Where would you put those statics and where would you redistribute them?

Option1: On your WAN routers. In this case if your ISP link fails, it will not make this static disappear from routing table and you will "black hole" your traffic. As a side note, statics will be removed from RIB only when interface leading to next hop address is down, it does not track next hop address itself.

Option2: On your Cable/T1 routers. It will work, but you need to run IGP all through your FireWalls,which might be non-secured setup.

IBGP provides elegant way to "import" your default route from external perimeter environment (Cable/T1 routers) to your internal trusted environment (WAN routers) via FireWalls which only need to allow TCP/179 or IP/47 (if using GRE) to make routing work.

We have a similar setup working in our environment and when we faced two options of using IGP all upto Internet routers or using IBGP, we easily picked IBGP and there was also no objection from our security guys....

David

[mbellears2]

Could you not use verify-availability(And tracking option) within the route-map?(Believe it came out in 12.3(4)T)

http://www.cisco.com/en/US/tech/tk364/technologies_configuration_example09186a0080211f5c.shtml

[mheusinger]

Hi,

about PBR: I do not agree about next hop behavior. The router will NOT forward towards unreachable next hop. It will just ignore PBR and "fall back" to normal routing table.

Second: a static route pointing to an interface will be withdrawn from the routing table in case the interface is down. No "black holing".

In case the interface is up the ONLY way to detect ISP failures is to do BGP peering, which is not an option here.

What I was talking about was how to get the default routes. This is one thing. The other thing is to propagate them to the WAN routers through the FWs. And here I agree with you, BGP is a valid and elegant option.

Hope this helps

Happy New Year!

Martin

[dknov]

Hi,

First, Michael is correct about verify-availability feature. Thanks, Michael. Cisco had come up with so many "small" enhancements to original features that it's hard to keep track unless you study by heart all the IOS release notes :-)

I am sorry Martin, but after conducting a small test I still stand behind what I said. If you specify in route-map statement "set ip next-hop X.X.X.X" and address X.X.X.X (doesn't have to be directly connected) becomes unavailable, however next-hop interface leading towards X.X.X.X stays up, router will NOT fallback to routing table, but drop traffic, i.e. blackhole it.

This new cool feature if tracking inside PBR addresses and resolves this issue, cause in this case SAA probe tracking X.X.X.X will fail taking down PBR and in this case router will indeed fallback to routing table.

Happy New Year to you too!

David

[mheusinger]

Hi,

maybe I was not clear enough about "unreachable". For me reachability means that there is an entry in the IP routing table. This is NO statement about connectivity. Connectivity means for me to be able to exchange IP packets. So a destination can be reachable from a routing perspective but there can be no connectivity (f.e. because of a firewall). So what I meant is: if a next hop is unreachable, i.e. no matching entry in the routers IP routing table then "fallback" will occur. I did not say anything about connectivity as this depends on much more than IP routing reachability. One example you gave is "interface up - no connectivity to next-hop". Still in your example the next hop is reachable from an IP routing perspective local to the router in question.

Maybe I am not using those terms in a proper sense. I am no native english speaker, so correct me in case I misunderstood something.

Happy New Year!

Martin

[corvallisclinic]

I am unclear how I could get IBGP to work correctly since the ISP does not support BGP peering. That is why I was looking at PBR with multiple tracking options. I would like to find a way to determine if the two ISP connections are up, and load balance between them when both are up (probably per destination). If one link fails all traffic is diverted to the other ISP entirely.

I have used BGP very little though, so if there is something I am missing, feel free to let me know!

Thanks again for all the ideas and help!