I am working on a project were internet service will be added to a backup/DR data center to service users in case the internet service at the primary data center goes down. The two data centers are connected through an ethernet service. Currently the network default route points to the inside interface of the primary site firewall. In the case that the Internet Circuit at the primary site fails, how can I configure the internal network so that the default route switches over to the inside interface of the firewall at the backup location without manual configuration. I would assume I would need to do some sort of SLA monitoring on the firewalls (ASAs) to detect any outages on the circuit. Can I add a backup default route to the layer 3 switches on the inside (Nexus 7K at primary site, 4500X at backup site) so that when the SLA monitor detects an outage the backup default route is inserted into the routing tables?
Are you running a routung protocol between the DCs ? Sounds like you are not but just wanted to check.
IP SLA is an option but if you run it on the firewalls and you are not running a routing protocol how will the internal switches know to send traffic to the other DC. You don't really want it having to bounce off the firewall.
Does the backup DC use the primary site for internet if the primary link is up ?
What would be better is if the default route used on the Nexus was tracked using IP SLA and if it was up it was redistributed into an IGP. The 4500 would then have a floating static pointing to the backup firewall but the AD would be higher that the redistributed route. So it would use the primary DC if the link was up. If the link fails it then uses it's own default route and advertises this into the IGP and all traffic switches to the backup DC
You could run tracking on the firewalls but you still need some way of letting the switches know to change the route so again an IGP would be useful.
But the above is making a lot of assumptions eg -
1) IP SLA is supported on N7K - easy to check
2) the N7K and the 4500X route the vlans for their respective DCs
3) the interconnect goes from the N7K to the 4500X
4) you are prepared to, or are already running an IGP
5) you would be happy to allow ICMP ping from the Nexus, out to the internet and back again
note it would only be the Nexus, no need to track the 4500 default route.
How is the current default route being propagated to the secondary DC or do you not need internet access from the secondary DC.
The only downside to the above is if the interconnect failed the 4500X would use it's own link for internet but that may not be an issue.
EIGRP is running between the two sites. Currently the backup site has a default route configured on its 4500 pointing back to the primary site. We would like to keep it that way and only change if the primary circuit at the primary site fails.
It would be better if you could use a default route on the 4500 pointing to it's own internet connection. It wouldn't use it the primary ISP link was up and the interconnect was up because it would prefer the default route being sent from the primary DC.
If the interconnect failed then would it matter if the secondary DC used it's own link for internet ie. it can't get to the primary DC anyway whether or not the primary ISP is up.
It really would make failover relatively simple.
I would still need answers to my other questions.
I back-up Jon's idea. In this case you have to use IP-SLA with object tracking both on N7K and 4500-X to switch their default-routes back to Back-up site/DC or vice-versa.
I might be missing something here but i think you only need to track the route on the N7K. Because if the floating static on the 4500X has a higher AD the 4500X will use the EIGRP route from the primary DC.
If the IP SLA on the Nexus fails then the default is removed and so is not redistributed into EIGRP and so the 4500X no longer gets it and installs it's own default route and then advertises it into EIGRP. If the primary link comes back up the Nexus installs it's default route into the IP routing table and advertises it and again the 4500X prefers that so removes it's own route and stops advertising.
I can't see the advantage of tracking on the 4500 because if the primary link is down it doesn't matter whether the IP SLA fails from the 4500, it has no ISP to failover to.
Am i missing something. It's been that sort of day so i wouldn't be surprised if i am
We could point the default route at the backup site to use it's own service, that is no problem. My concern is what if the backup circuit failed, can the backup site then failover and use the primary as the default. This would be if for some reason something in the backup site is needing internet service for testing or some other reason. Jon what other questions were you needing answered?
You point the 4500X to the backup link with the default route but if the primary link is up it doesn't use the backup link, it uses the primary because it prefers the route it got from EIGRP than the one configured on it ie. the default route on the 4500X would have an AD > 170 and the default route it receives from the Nexus will be AD 170 so it goes via the Nexus.
So it's not so the 4500X uses the backup link even when the primary is up, it is so we can get the failover working properly.
Other questions are pretty much 1 - 3 and 5 from the above. I can check 1) if you tell me the code version but as Amit has suggested using IP SLA as well i suspect it is supported.
Couple of questions about EIGRP setup. This solution requires redistributing statics into EIGRP so -
1) are you currently redistributing or using things like "default information-orginate" under EIGRP to advertise the default route into EIGRP or is it just statics you are using.
What i mean is does EIGRP advertise the default route from either switch or not ?
2) does either the N7k or the 4500X have any other statics (other than the default routes) that you would not want redistributing into EIGRP ?
The interconnect. The assumption we are making is that the interconnect runs between the N7k and the 4500X and is L3 terminated on those switches either with L3 ports or SVIs.
What i mean by that is that if you tracerouted from the primary DC to the secondary DC firewall then a L3 hop would be the 4500X and vice versa. So within each DC everything is routed off their respective switch.
Apologies for all the questions, it's just that we do not want to supply you with a solution that does not work or could create problems in your network.
I have used a lot of assumption here as we are without a network diagram. My mistake probably would have been not to read the last line carefully from the poster.
Firstly, I thought that we have FW's connected directly to both N7K and 4500 when he mentioned the ethernet connection between the DC's. I thought its a switched L2 connection not an L3 connection so we are extending vlans for L2 hearbeat between the Fw's.
Secondly, I thought that I would use floating static route without redistribution at each site both tracking the primary link using a local object in each site. When the primary goes down and my object fails, my both the core devices start pointing to a secondary/back-up firewall. That's why I mentioned using tracking on both boxes with the local objects being tracked at each site.
Its my mistake that I did not think about how would I make my floating static route into the EIGRP routing tale for others to use it as a gateway. My bad, i should have read it properly before jumping...
See, i am getting a bit old now ...:-(
Yes, IP-SLA with EOT is supported on N7K. You should be ok with that.
Can i just confirm. There may need to be a route map used with the redistribution anyway if there are other statics that shouldn't be redistributed but if there aren't i seem to remember reading somewhere that with Nexus you have to use route map with EIGRP to redistribute any statics.
Is that the case ?
Also i may very well need your help with IP SLA if it is different on the Nexus switches as well.
Better yet, i may just leave the entire Nexus stuff to you.
I really need to get a job where i can get my hands on these
Wait till you are old as me
That's one of the outstanding questions ie. does the interconnect go direct between each switch ie. it might not work if the interconnect went to other devices like firewalls first.
Yes, that's what I thought that L2 interconnection is between the boxes directly.
Yes, you need to use the route-map for route redistribution.
Wait for Cisco VIRL to get started shipping. You can use NX-OS, IOS,IOS XRV and IOS-XE devices to run your topology.
Thanks for the responses. I am still gathering information on this as well. The interconnect between the two sites does use L3 SVI's. The plan in the future is to run OTV between the two to strech the L2 domains. A route map is already in place to redistribute certain static routes to the backup site.
If you want to use the proposed solution in the meantime just let me but if you are going to be using OTV i suspect you might need a different solution. The solution i propose would work if the N7k and 4500X were peering with EIGRP and for any interconnect traffic the first device they get to is either the N7k or te 4500X.
Amit may be able to help with that as i have no experience of using OTV.
Having OTV between the sites does not change much from the overall L3 routing toplogy. The only thing that OTV will bring in this architecture is a bit complexity :-). Not that much but your stretched L2 vlans across site will have to be designed for a proper Egress routing. Either you would end up bringing all the traffic back to main DC and then route it out to the outside world or to some HSRP filtering and let the extended vlans take an egress path from the DR site.
If you are using some L3 BGP peering on the WAN this would complicate the design as to from which site you would adverstise the IP subnet.
We will work with you when you get to that stage and i dont want to over say few thing about the future here. I would be more than happy to assit you with the design and Jon with the troubleshooting :-).
Hope this helps.
I would be more than happy to assit you with the design and Jon with the troubleshooting :-).
Volunteering my services now
Seriously, happy to help out any way i can.
Quick question Amit. I am, funnily enough, just reading a document on OTV.
Does OTV allow you to extend some vlans and route others ?
If so how is the join interface related to the actual physical links used to connect the DCs ie.
if the join interface terminates the actual physical link then what does it do with normal routed traffic as there is no need to encapsulate it.
Yes, OTV allows you to extend Layer 2 vlans to the remote site. That being said, OTV is typically used as an appliance model design. You stick either OTV VDC on N7K or just Cisco ASR1000 as the dedicated appliance. On N7K, the restriction is that you cannot have the OTV in the same VDC as the SVI hosted for your Vlan traffic. So you stretch your L2 trunks to OTV internal-ports and exit the extended Vlan traffic out of the "Join-Interface" which an L3 port connected back to your routed core network.
OTV join-interface encapsulates (MAC in IP) the MAC's to be forwarded to the other side and forwards it over the OTV overlay. The "join-interface" is just for the encapsulation, as long as its routed and reachable, you would be able to have a OTV tunnel established properly. In current designs on N7K, your join-interface can be a dedicated L3 routed port for Inter-DC connectivity or an L3 routed port connected back to the core router/VDC. Customer prefers the second option of having the join-interface connect back to the routed core just to save extra cost of deploying a dedicated L3 port.
The normal routed traffic is handled by the device hosting your SVI's. The traffic that needs to be routed goes stratight to your SVI/L3 gateway and exit out. The traffic that needs the L2 forwarding to the extended vlans,hits the OTV appliance internal interfaces, get encapsulated over the overlay and get to the remote side depending upon the OTV mac routes existance.
OTV can be deployed as Multicast model or Unicast model depending how the WAN core looks like related to multcast.
Hope this helps.