BGP multihoming in two locations with multiple ISPs

[Iain]

Hello,

I have a question for which I have not yet been able to get a satisfactory answer. Thus far I haven't been able to talk to anyone who has experience with multi homing BGP in separate geographic locations with separate ISPs. Any assistance you might be able to offer would be greatly appreciated!

What we have (current config):

We are a University with two locations (main campus in the state of Missouri and sister campus in the state of Arizona).

Both sites have their own ISP (through dissimilar providers)

The sites are connected by an internal WAN link (long haul circuit)

The private network of the Missouri campus can be summarized as 10.1.0.0 /16

The private network of the Arizona campus can be summarized as 10.2.0.0 /16

We are running NAT with PIX 515e firewalls

The networks are setup with a hub and spoke design (single Cat6509s at each hub)

Public IP Addressing

Missouri campus has /26

Arizona campus has /24

What we want to do in the near future:

Our University is predicting significant growth in the area of online education. Therefore the reachability of our online resources (mostly web content) will be much more critical. To address these needs, we plan to mirror web content between both sites. The goal is to be able to sustain a complete ISP failure at either campus location. It seems we will need to get a new addressing space, an AS, and start running BGP.

However, there are several pressing design-related questions which I have not been able to deal with.

1.) If my web servers are behind a firewall / NAT, how will the BGP edge router be aware of changes in reachability of these NATed addresses? For example: In the event of a firewall failure the entire server subnet would become unreachable. I want to be able to ensure that there is enough sensitivity built-in to the routing design that if the server subnet (behind the firewall) becomes unreachable it will trigger a change in the routing table.

2.) This is a more basic, but where do I start when looking for a new address space? Do I go directly to ARIN? Again, any practical advice in this area would be extremely helpful.

3.) What makes the most sense as far as traffic flow? Is load balancing feasable in such a senario?

Thanks, Iain

PS See attached network diagram.

[AJAZ NAWAZ]

Ian,

Hi, let me take a stab at qtn #1... It's just a one solution that could propabably work but no doubt there may be better ways to satisfy the requirement.

You could have a pair of routers sitting behind the 515's iBGP peering with the bgp 2800/Tazman and internal mpls enabled 2800's . Advertise the outside static addresses from the new routers through the firwalls. If the firewall fails the bgp peering goes down and so the routes arn't advertised any longer. You iBGP setup between both sites will allow you to be deterministic regarding which path inbound traffic should arrive at. Further, get both ISP's at the table and tell them about your requirement - they may offer a better solution. You'll have to speak to them anyway since bgp policy changes may be inevitable. Of course this mean allowing bgp through the firewalls but that's well documented.

With regards to question #3, if one of the sites is intended to be back-up (aka DR) then I would avoid l/blcin'g between sites simply to avoid the complexities. Try to keep it simple as possible at this stage.

hth

Ajaz Nawaz

[sean]

Your BGP edge router would have to participate in your EIGRP process. Your IGP routes will need to be redistributed into your BGP process for advertisement. Synchonization would also need to be on in BGP. If the firewall is inline between your BGP edge router and your eigrp neighbor, then when this dies, eigrp will no longer redistribute into BGP. If you wanted to grow each site with their own independent control, you could create a BGP confederation and use intra-confederation eBGP peering between your two sites.

You would go to http://arin.net/ for your IPs and your ASN.

You could manual create load balancing, but you might cause more harm. When multi-homing, the goal is for redundancy. Let BGP take care of the best path choice. Your links might see an even balance, or one link might receive more traffic than the other. Its just a question of which provider has the best peering.

Hope this helps.

[Danilo Dy]

Whatever you do, don't put stateful firewall in the iBGP path. If you are multihoming, the incoming/outgoing traffic in your BGP speaking devices to your upstream ISPs (eBGP) is assymetric.

[ndarnell]

Looking at it from a simple perspective one option would be to run Enhanced object tracking on the edge routers pinging the DMZ interface on the Pix - should it fail a route map can be used to change the local preference setting the prefered route to the other site.

Rgds

N

[mark.edwards]

1. There are many ways to achieve this. As already mentioned running some sort of routing protocol through the FW is quite normal. Can be BGP or an IGP.

2. You could get IP space from your ISP. The only issue hee is if you ever change ISP you would need to make changes to reflect new IP ranges (DNS, FW NAT etc).The other option is I guess would be ARIN but I'm not sure what the current criteria for obtaining your own block is.

3. I don't see why you should not load balanace. If you are doing loadbalancing keep in mind if you need persistance enabled for your web site as this may have an effect on how your design works!

[sachin]

Hi folks,

Have you given thought over GSLB (Global Server Load Balancing )?

[sachin]

Pls check this

http://www.cisco.com/warp/public/117/css_glb_sticky.html

[Danilo Dy]

You can do similar round-robin using DNS for any service (SMTP, WEB, etc..)

1. 'A' records for member servers

www1.domain.com. IN A 192.168.1.1

www2.domain.com. IN A 192.168.2.1

smtp1.domain.com. IN A 172.16.1.1

smtp2.domain.com. IN A 172.16.2.1

2. 'A' records for load balancing

http://www.domain.com. 180 IN A 192.168.1.1

http://www.domain.com. 180 IN A 192.168.2.1

smtp.domain.com. 180 IN A 172.16.1.1

smtp.domain.com. 180 IN A 172.16.2.1

3. 'PTR' records for member servers

1.1.168.192.in-addr.arpa. IN PTR www1.domain.com.

1.2.168.192.in-addr.arpa. IN PTR www2.domain.com.

1.1.16.172.in-addr.arpa. IN PTR smtp1.domain.com.

1.2.16.172.in-addr.arpa. IN PTR smtp2.domain.com.

Use http://www.domain.com to access the web server. Use smtp.domain.com to access the smtp server.

[nguyenphu]

Hi,

I think it can help to U :

Service Assurance Agent(SAA)

nguyen