Quadruple WAN links from Two ISP's

Gerard Gacusan · ‎07-09-2009

Scenario:

Quad-WAN links between main site and remote site.

Two routers on both ends:

Main Site:

R1 :

1 x 1Gig via Metro Ethernet (ISP1)

1 x DS3 via MPLS VPN cloud (ISP2)

R2 :

1 x 1Gig via Metro Ethernet (ISP1)

1 x DS3 via MPLS VPN cloud (ISP2)

++++++++++++++++++++++++++++++++++++

Remote Site:

R3:

1 x 1Gig via Metro Ethernet (ISP1)

1 x DS3 via MPLS cloud (ISP2)

R4:

1 x 1Gig via Metro Ethernet (ISP1)

1 x DS3 via MPLS cloud (ISP2)

R1 To R3

R2 To R4

+++++++++++++++++++++++++++++++++++++++

ISP1 is a metro ethernet layer-2 tunneling and I believe this is a default-route only or static. ISP2 runs in a single BGP AS/Frame-Relay over to an MPLS cloud.

Voice/Video/Data are passing these 4 links. Redundancy, load-balancing and/or load-sharing is necessary.

Lot of considerations here - PfR, OER, CEF's, PBR, GEC, GLBP. Any inputs howto achieve perfect redundancy/load-sharing/load-balancing scenario? MLPPP and/or MFR is not an option on DS3's at this point.

Thanks in advance ...

Giuseppe Larosa · ‎07-10-2009

Hello Gerard,

you have four links but they are of quite different speed DS3 45 Mbps and full GE is 1000 Mbps.

DS3 links are provided with a L3 VPN service.

GE links are provided with L2 service.

So first of all:

on primary GE links you can run your own routing protocol that you like with the other side for example I recommend OSPF for its clear hierarchy of route types.

These provide O or O IA internal routes.

DS3 links may use eBGP to connect to the PE nodes: routes coming from other side can be redistributed into OSPF.

Downstream routers at each site will prefer internal routes over external routes of any type.

For the edge routers the scenario is different a single prefix like 10.10.10/24 has AD 110 if received in an OSPF update and AD 20 when received on eBGP session.

Two measures to use primary GE links on edge routers:

use of network commands + aggregate address to create less specific routes;

AD change manipulation so that eBGP routes are assigned an AD > 110 with

bgp distance command.

I would do both for more safety.

With all these choices DS3 wouldn't be used until both primary GE link fails.

If you want to divert some traffic over DS3 in normal conditions you can use PBR on edge routers on both sides.

If there are downstream / inner routers on each site they can use OSPF to see two equal cost paths to Rx Ry.

If there are client Vlans directly connected to edge routers Rx and Ry GLBP can be handy for them but only if there are multiple clients: GLBP load balancing works on n.client > 1

CEF per destination load balancing should fit your needs on both primary/secondary links.

As you see I'm proposing a hierarchical scheme with GE links used first, DS3 working as backup with the possibility to have some specific traffic flows diverted to DS3 links.

Given the different type of service it is difficult to use all 4 links only with dynamic routing.

Hope to help

Giuseppe

Gerard Gacusan · ‎07-13-2009

thanks for your inputs. yeah, i can do PBR's to route specific traffic to some links or BGP next hop selection. Trying to figure out how i can use all of them in a load-balancing/sharing fashion from different links.

Joseph W. Doherty · ‎07-10-2009

Like Giuseppe, since there's such a disparity in bandwidth between the gig links and the DS3, the simple solution would be to not have traffic normally flow across them (DS3s) unless both gig links fail. If you want to use them for some traffic, and if your platform supports it, I would recommend using OER/PfR. Also, OER/PfR would provide about the (2nd) best possible load balancing for the two gig links, with or without directing traffic to the DS3s. (Unless the gigs are very busy, or down, OER/PfR would likely not direct traffic to the DS3s.)

For initial load balancing, you either want to split the load of traffic across the two WAN routers, GLBP, mHSRP, LAN router(s), would be options, or you could direct traffic to just one WAN router and have it split the traffic.

An extreme balancing option would be to use CEF packet-by-packet. Because of both you having more than 2 links and more than just TCP traffic, this is something you should avoid. (Note, if you had a need to provide some TCP traffic 2 gig of bandwidth, a very, very involved set-up could provide it.)