Interent Redundacy with BGP and GLBP or HSRP

[jahetrick]

I've been tasked with designing Internet redundancy into and out of our network with our single ISP (I know two ISPs would be better). I've been doing some research and thought I had a good grasp on things, but now I'm confused on a few things. Lots of info, sorry for the long read.  Any suggestions are appreciated.

Setup:

The ISP will provide two 50Mb links
Each link will go to a geographically separate edge router on my network
The ISP will only be providing default routes to my edge routers.
The ISP will be providing me a private AS # and two /30 public IPs, one for each of my edge router's "WANs" that will be connecting to the ISP
The ISP already provides me with two public address blocks (for argument sake lets call them 64.x and 65.x) that sit on the "LAN" side of my edge routers

Each edge router will connect to a layer 2 edge switch on the LAN side

The edge switches will connect to each other, providing a "LAN" link between the edge routers

The main goal is inbound/outbound redundancy
A secondary goal is load balancing
I know load balacing can be a whole topic in itself, if we do load balancing, my main priority would be load balancing outbound towards ISP, I don't really care about inbound from ISP

See a rough sketch below:

MyISP-RtrA                    MyISP-RtrB

     |                                      |

     | "eBGP"                          | "eBGP"

     |                                      |
MyEdge-RtrA                 MyEdge-RtrB

     |                                        |

     | "Sub-ints, 802.1Q"            | "Sub-ints, 802.1Q"

     |                                        |
MyEdge-SwA------------------MyEdge-SwB
    |                                          |
    |                                          |
VLAN 64: 64.x.x.x       VLAN 64: 64.x.x.x
VLAN 65: 65.x.x.x       VLAN 65: 65.x.x.x

Questions/Concerns:

iBGP, IGP, HSRP/GLBP on the LAN side of my edge routers.  I'm reading conflicing suggestions and having issues deciding what to use and in what combination.

1. Assuming no other routers exist on the LAN side other than my edge routers, is there any reason I should use iBGP or an IGP (OSPF) in this setup? Only advantage I see in doing so would be that if Myedge-RtrA looses its link to MyISP-RtrA, then it would know to use MyISP-RtrB's link and vice versa.

2. If I would use iBGP, is there any reason to still use an IGP like OSPF with it? Again assuming no other routers exist on the LAN

3. To facilitate first hop outbound redundancy, I want to use either HSRP or GLBP on the LAN side of my edge routers.In either instance I would use IP SLA with object traffic to avoid outgoing blackholes if the WAN link is up on my edge routers, but there is an upstream problem in the ISP. Any issues?

4. I'm worried about blackholes on my edge router LANs from an incoming BGP standpoint. For example if the link between my edge switches fail, but each local edge switch is up and its link to the local edge router is up, I think my edge routers will still advertize that they each have a route to the LANs. If an inbound packet destined to a host that lives on the 64.x network behind MyEdge-SwA, comes in through the MyISP-RtrB link, MyEdge-RtrB will think it has a route to that host, but due to the down link between my edge switches, the packet will never get there.  Could I get around this with some kind of IP SLA and object.

Thanks!

[smitesh kharecha]

Hi Jonathan,

Below are my inputs to your questions:

1. No there is no need to run iBGP or IGP in the network. The redundancy goal which you want to do can also be done via static routing or even bgp with advertising lower med for network the  network which is on other router.

2. IGP is generally used in conjuction with iBGP where there is requirement on NOT to do full mesh. Since in  your topology, their is only two routers involved, if would be full mesh only. So no need to run iBGP (only thing which needs to be confirmed is both router/switches should know about all the routes).

3. Yes HSRP or GLBP  with track is perfect solution.

4. I think this is tricky. If your are advertising the network via redistributed connected in BGP, this won't be the issue; becasue, as soon as you loose your LAN link, those network won't be advisertised to your peer (BGP requires the routes to be reachable, before it advertises to its neighbours).

HTH,

Smitesh

[jahetrick]

Hi Smitesh,

Thank you very much for your answers.  That really helped clear things up for me.

I know what you mean about #4 being tricky. Maybe its my lack of in depth understanding of BGP, but I’m not sure that even if I were advertising the routes via redistribute connected that would avoid the issue. Since in the scenario I posed, each router’s LAN would still be up and active, as each switch is up and active, just the link between the switches is down.

Thanks again!

[smitesh kharecha]

Hi Jonathan,

Not too sure, but putting keepalives on LAN interface can help you bringing that interface down, if any issues on that port.

So basically, keepalive will keep track of keepalive from LAN end. When it misses keepalives (as configured) it will bring the port state down and hence, even the connected route from the routing stable will be gone. Once that route is gone, it will no onger advertised to the neighbour.

Regards,

Smitesh

[neil grant]

Hello Jonathan,

So this is purely a Internet solution, I will start with the solution I have experience of which has proved itself. 

BGP into OSPF - Redistributing the only the DFR,

A firewall or router attached peering with both via OSPF on the same lan (see's 2 equal paths to 0.0.0.0 and loadbalences on the egress)

ISP needs to enable BGP multipath to you 64.x 65.x at there AS boundarys with you advertising networks equally, this enables internet ingress traffic to utillise both links towards your public addresses.

the above achieves the load balancing you are seeking,

With regards to redundancy you would want the FW in a Cluster / Router HSRP (ideally FW) one at each of the 2 sites (tied to tracking statments either IPSLA or protocol.

This I think you work best for you and remains elegent. And i would really push for the extra buget to get this working. and scale well (if you can afford 2X50Mb.....)

If you only have the kit listed at your disposal you will be looking at a GLBP, which i have never used/tester so cant comment any further.

Neil