11-07-2011 09:14 AM - edited 03-04-2019 02:11 PM
Hi All,
I'm looking to add a second data centre to our network for DR purposes and I need to design the Internet connectivity so I can failover the public IP addresses to the DR site when required. See diagram.
We currently have 2 internet edge routers which use eBGP to 2 ISPs, iBGP is configured between the edge routers. We have our own ASN and IP prefix and we prepend the ASN over ISP2 to prefer the route via ISP1
We are looking to add a single internet edge router in the DR data centre and again use eBGP to the 2 ISPs. We will then advertise the primary prefix out from the DR data centre and prepend ASN again to to make this route less prefered then the primary data centre.
For this configuration to work, will I need to use iBGP between the DR data centre router and the Primary? If so will I need to use a dedicated link between the edge routers or can I connect them via Internet. Also any throughts of using a Cisco 3750 as the edge router in the DR data centre? We will only be receiving default routes from the ISPs.
Thanks
11-07-2011 06:29 PM
Hi
from a high level point of view this is doable design
but there are some points you need to test, discuss and take it into your considerations
i am assuming the DR wil be just a redundant/Back up services that dose not need to communicate with the primary DC and they will be use din the case of the primary DC is down if this is correct then you do not need a link with iBGP between the two DCs
if the services they require some communications, like backup, replications ..etc a link between those two data center it recommended to avoid routing a larg amount of traffic over the WAN
however, adding a link between the two DCs might lead to advertise the primary DC over the DR link when the eBGP of the primary is down and this will lead to make the primary DC comunicate with the Internet over the inter DC link ( some BGP policies can be configured to overcome this issue )
if you are going to use same BGP AS number or diffrent AS number you will need to make sure that both ISPs accept AS prepedning, the advertisement of the IP range you have over all links and confirm with each ISP regarding which link, to be preferred within thier cloud as some ISP they use some BGP community stings significant within the ISP cloud and their customers and might lead to some sub optimal routing ( discuss it with each ISP )
let me know if you need anymore details about any of the points above
hope this help
if helpful Rate
11-08-2011 12:58 AM
if you'd like to use the second DC for completly separate services, then may be it's better to get an additionl AS for it.
If you'd like to have all DC's in the same AS you'll need to interconnect both DC via iBGP, you can do it withj help of dedicated link or you can look int some kind of GRE tunneling, if it's supported on your hardware. And you should be carefull in advertising of networks from differemt DC - you want that incoming traffic goes to coreect DC and router.
11-08-2011 01:26 AM
Hi,
There will be a dedicated fiber link between the core switches in each DC which will be used for replication traffic etc.
The second data centre will be used for test and dev (seperate dev IP address range) but I would like to failover the primary data centre IP address to the backup data centre in the event of a large failure. All of the servers will be in a VMware enviroment so they will be started up in the backup data centre.
If I need iBGP between data centres, its is advisable to use the dedicated fiber between the intenal core switches (using a dedicated VLAN) or setting up a GRE tunnel over the internet between the edge routers? I dont think I will have buget to have 2 dedicated links between data centres.
To avoid any routing issues, I suppose we could make this a manual process by advertising the live IP address range on the backup data centre router in the event of a failure?
Thanks all
11-08-2011 01:42 AM
if you are advertising differnt range then this is not a DR case anymore
however if you plaining to faiover the VM environment over the L2 link in this case you can have automated failover rather than manual by using the L2 vlan used over the fiber as L3 interface and exchange routing over it
by the way you talking about a failover from ESXi and VM point of view where the VMs will be moved using DRS/vmotion for example from the main DC to the DR
while you might have a situation where the routers, links or ISPs have an outage which will cause only the Internet link to go down and here where you can take the advantage of runing iBGP between the two DCs and advertise the Primary DC over the DR eBGP peering for backup reasons
hope this help
11-08-2011 02:01 AM
I can't say much about ESX vmotion ( i had in the past a little bit experience but so much).
For iBGP I'd suggest to use to dedicated link with two separate VLANs to connect both routers and a GRE tunnel as a backup for the dedicated link.
If your dedicated link is a "flat" and very fast then actually you don't need to think much about the routing, it doesn't really metter how the traffic comes in and out, if for example, the latency between two DC is about 5 ms.
But I don't recomend to rely on manual process - it's better to sit and plan a dynamic failover routing schema carefully.
11-08-2011 04:24 AM
Hi Guys,
Thanks for your comments, very helpful.
The Virtual Machines will only move to the DR data centre during a complete outage, this will happen using VMware Site Recovery Manager. This means that we will have a small amount of time to move the live IP address range to the DR site whilst the Virtual Machines are coming up. We dont need instant failover.
That is a good point regading an ISP outage in the primary data centre. We wont want to initiate DR just because the ISPs are down if we can help it, so routing the live IP address range via the backup dc and over the fiber link using iBGP to the prmary will be good.
In order to connect the edge routers using iBGP, it is advisable to use the flat fiber connection between the internal core switches by creating dedicated VLANs on the core switches and then directly connecting a port on the core to the edge routers, this will place the iBGP routers in the same broadcast domain and we can peer that way? Or we can connect the edge routers between sites using an IGP over the internal network and GRE tunneling?
Thanks
11-08-2011 04:35 AM
willscotty2012 wrote:
In order to connect the edge routers using iBGP, it is advisable to use the flat fiber connection between the internal core switches by creating dedicated VLANs on the core switches and then directly connecting a port on the core to the edge routers, this will place the iBGP routers in the same broadcast domain and we can peer that way? Or we can connect the edge routers between sites using an IGP over the internal network and GRE tunneling?
exactly, it's a good idea to connect the dedicated link to the core switch, but I'd suggest to use 2 separate VLANs for connecting between DC's, first it will be easier to manage the traffic flow (costs, filter and so on), second it's a protect from problem inside of a VLAN.
The iBGP router shouldn't be in same broadcast domai in order to get "connected". But if you'd like to be 100% sure how the traffic is flow between BGP routers it's better to connect them directly.
11-08-2011 10:17 AM
I have just been told that GSS might be a better solution here.
GSS will respond with the primary data centre IP address in its DNS replies during normal operation, and the backup data centre IP address in the event that the primary site is down.
Has anyone had experience/problems with using GSS in a active/standby scenario?
11-08-2011 02:04 PM
It depends what type if services you use
GSS us commonly used in scenarios like yours but keep on mind that if you have GSS located in one DC you will have same issue reaching the GSS
You need to gave two redundant GSS one in each DC
Two hosts dns records in the ISP for failover one point to the primary GSS and the other failover dns record point to the secondary GSS
Or you can have one GSS in the main DC but by using some routing between the two DCs you have it reachable via both ISPs and DCs
The other option you have two loadbalancers in each DC and use the dns concept described above
Cisco ACE is one option
Hope this help
Sent from Cisco Technical Support iPhone App
11-08-2011 03:08 PM
Thanks for the reply. Have you had experience with GSS? Which solution, BGP routing or GSS, in your opinion would be more robust, easier to manage etc?
Thank you
11-08-2011 03:47 PM
well the is depends on your needs and equipments
if you have loadbalncers in both sites GSS is a god option to go with
if not
routing can do but you need to consider all the failover scenarios, server, VM, ISP, router, Link .etc
and address all of these cases with your design
i believe the easy option is routing by using some BGP polices, however you need to sort out the IP range advertisement issue so that you can smoothly advertise and failover to the DR DC
HTH
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide