I am working on a multi-home design accross two datacenters. We have an ARIN assigned /24 public space and a single AS#. We would like to carve up the /24 to be used in both DCs. Of course our ISPs will only accept a /24 as a minimum advertisement. Is there are way acomplish BGP redundancy and still split the /24 into two /25 to be used at each DC? From what I am reading we can use AS prepend, but will that work if I only prepend a /25? It hasn't worked on the lab.
Thanks in advance.
| BGP |
You could carve up the /24 internaly if you still advertise it as /24. What's the goal of the design, one DC is primary and one DC is purely secondary? There should be no load sharing?
You would need BGP between the DCs. How far is it between them? Would it be acceptable to receive traffic for DC1 coming via DC2? That's the issue you will have if you don't have larger address space. So the traffic would be sent over BGP between the DCs. Even if you do AS prepending or setting MED or communities etc there is no guarantee that you won't receive incoming traffic on DC2 even when DC1 is up.
You could do some form of BGP conditional advertisements or use IP SLA, EEM and so on but it's your choice if you consider that to be an acceptable design.
Please rate helpful posts.
The question Daniel brought up is a good one. Is this for load sharing, or redundancy?
If your ISP is only going to accept a minimum of /24, then obviously you can't advertise out the two /25 subnets of the /24.
So let's say you assigne 184.108.40.206/25 to DC-A and 220.127.116.11/25 to DC-B. If I buy a Web Server from you so to speak, and have a website called www.awesomesauce.com, which points to that webserver, then from the Internet's point of view, it's most likely going to frst find the larger block of that space, tha your ISP is announcing most likely like a /21 or /22. And depending on how big the ISP is, it might be part of another ISPs larger block.
Also, depending on how the AS_PATH or other attributes, one ISP may be considered better than the other, so to get to 18.104.22.168, may always go to DC-A, and then if there is a link to DC-B (I would assume), it would go to DC-B then the Webserver at 22.214.171.124. This would obviously be suboptimal routing.
I honstly don't think there is a good way to do this, without causing any weird issues.
The only way would be to configure routing fo the /25s ont he CE going to the PE equipment, for the respective /25 network.
Thnks for the input. This is for redundancy, but we would like to use the address space on both DCs. In a way we want to use it as active/active and for redundancy. Each DC has different traffic, and we only want it failover in case of an ISP failure, but we also want to utilize the IP space at both locations.
There will be iBGP between both routers. I was thinking of doing PBR on incoming traffic for each /25 and AS prepend for outgoing routes. Will this even work? Is it a good design?
Thanks for the input Jose.
Jose, is this provider independent space or provider provided address space? And, since from my understanding, (correct me if I'm wrong), you want for example, DC-A gets 126.96.36.199/25 and DC-B gets 188.8.131.52/25. This is going to be hard to do both.
If you advertise out the /24 from both Datacenter routers, incoming traffic is going to match a /24 and not a /25, so if someone wants to get to 184.108.40.206, which is at DC-B, and you have AS_PATH prepending on the 220.127.116.11/24 going out of DC-B, it's going to goin at DC-A, and then if it has iBGP between them both, to DC-B, which would work, but would be suboptimal if you get what I"m saying.
Now, if you have network devices in the 18.104.22.168/25 range, then you could configure a default route, etc etc, so if it's at DC-B to go OUT at DC-B.
Also, are you going to be accepting the full Internet table or a default route?
This is independent address space. We will be receiving only default routes, so for outgoing traffic everything should flow properly. My concern is incoming traffic. I am ok with some traffic going over the iBGP connection between the two routers, but I would like for the preffered path for incoming trafffic to be the correct DC.
Does this make sense?
Thanks for your help!
You said that you have different services though. So maybe some services are primary in DC1 and some are primary in DC2? That wouldn't work. If you want to all traffic for that /24 to go primarily to DC1 then you should be able to do that but there are no guarantees. Hopefully you will not receive too much traffic to the "wrong" DC.
Have you looked into if your ISPs support communities? If they do you can usually set a community so that they will prepend their AS. There are also usually traffic engineering stuff like, do not announce to Europe, do not announce to US, only announce to peers and things like that. Might be worth looking into as well.
Please rate helpful posts.
That makes complete sense, jose, I'm just not sure how you can actually do they in your current enviornment.
The only way I can think of is that traffic to DC-B will mostlikely be going from DC-A to iBGP link between them, o DC-B, and then routed appropriately.
How fast will the iBGP link be between datacenters?
1. I would then, advertise out of DC-B the /24, but with AS_PATH prepend, to make sure it's not used as the primary link.
2. I would have all traffic, going to the CE router at DC-A, and then have routing setup for the /25 for DC-B to go over the iBGP link, and into that CE.
3. If the ISP at DC-A goes down, it will get to the /24 advertised out of DC-B, and be able to go from that CE router, and then have routing setup to go to the /25 at DC-A
4. If you have network devices at DC-B, they can still be used, and get Internet access going out DC-B (obviously) but return traffic will be asymetric, so you may have to tune your firewalls.
Since you can't do OTV, and or advertise out /25 from each DC, I think that design is your best choice unless someone else has something different.
I think that may be my only option, but thinking through the failure scenarios, if the 1Gig link goes down traffic to DC2 would be blacked holed. I may have to go ARIN and ask for more space.
As mentioned i think prepend is the way forward. For example, you have the Class C Subnet (22.214.171.124/24), and you are advertising this subnet through both ISPs and prepending it on ISP used as redundant link to look worse, or maybe you could Spilit the Subnet and advertise more specific Subnet our your primary ISP to look better from the Internet a /25 out your primary link and less specific out your backup to give you the required redundancy you need.
Thanks everyone for all the input. We've decided to go back to ARIN and request a /22 in order to make this work.
Thanks for your time!
The only thing i would add is that your interconnect is a single point of failure that even with a /22 could still isolate a DC depending on how the actual physical topology looks like.
If the devices that the interconnect is terminated on are in the direct path between the ISP and your firewalls then a failure of the device in say DC2 will make DC2 unreachable ie.
1) you cannot connect direct into DC2 because the device has failed and there is no path to the firewalls
2) you cannot connect via DC1 because the interconnect is down
The above may not relevant if the interconnect devices are not in the path from the ISPs to the firewalls. If they are though you can either -
1) add another interconnect. If L2 terminate on switch stacks/VSS pair etc. If L3 terminate on separate L3 devices or again a VSS pair. Obviously this could get expensive.
2) move the devices that are used for the interconnect out of the direct path so that the firewalls can still get to the internet even if the interconnect device is down. That way if the interconnect or one of the devices used for the innterconnect fails the DCs are isolated from each via the interconnect but both your DCs are still accessible from the internet.
If they are not in the direct path then please feel free to ignore all of the above