Routing for backup Internet service

Unanswered Question
Jan 20th, 2014
User Badges:

I am working on a project were internet service will be added to a backup/DR data center to service users in case the internet service at the primary data center goes down.  The two data centers are connected through an ethernet service.  Currently the network default route points to the inside interface of the primary site firewall.  In the case that the Internet Circuit at the primary site fails, how can I configure the internal network so that the default route switches over to the inside interface of the firewall at the backup location without manual configuration.  I would assume I would need to do some sort of SLA monitoring on the firewalls (ASAs) to detect any outages on the circuit.  Can I add a backup default route to the layer 3 switches on the inside (Nexus 7K at primary site, 4500X at backup site) so that when the SLA monitor detects an outage the backup default route is inserted into the routing tables?

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Jon Marshall Mon, 01/20/2014 - 12:44
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


Are you running a routung protocol between the DCs ? Sounds like you are not but just wanted to check.


IP SLA is an option but if you run it on the firewalls and you are not running a routing protocol how will the internal switches know to send traffic to the other DC. You don't really want it having to bounce off the firewall.


Does the backup DC use the primary site for internet if the primary link is up ?


What would be better is if the default route used on the Nexus was tracked using IP SLA and if it was up it was redistributed into an IGP. The 4500 would then have a floating static pointing to the backup firewall but the AD would be higher that the redistributed route. So it would use the primary DC if the link was up. If the link fails it then uses it's own default route and advertises this into the IGP and all traffic switches to the backup DC


You could run tracking on the firewalls but you still need some way of letting the switches know to change the route so again an IGP would be useful.


But the above is making a lot of assumptions eg -


1) IP SLA is supported on N7K - easy to check


2) the N7K and the 4500X route the vlans for their respective DCs


3) the interconnect goes from the N7K to the 4500X


4) you are prepared to, or are already running an IGP


5) you would be happy to allow ICMP ping from the Nexus, out to the internet and back again


note it would only be the Nexus, no need to track the 4500 default route.


How is the current default route being propagated to the secondary DC or do you not need internet access from the secondary DC.


The only downside to the above is if the interconnect failed the 4500X would use it's own link for internet but that may not be an issue.


Jon

Mitchell Theriot Mon, 01/20/2014 - 12:48
User Badges:

EIGRP is running between the two sites.  Currently the backup site has a default route configured on its 4500 pointing back to the primary site.  We would like to keep it that way and only change if the primary circuit at the primary site fails.

Jon Marshall Mon, 01/20/2014 - 12:52
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


It would be better if you could use a default route on the 4500 pointing to it's own internet connection. It wouldn't use it the primary ISP link was up and the interconnect was up because it would prefer the default route being sent from the primary DC.


If the interconnect failed then would it matter if the secondary DC used it's own link for internet ie. it can't get to the primary DC anyway whether or not the primary ISP is up.


It really would make failover relatively simple.


I would still need answers to my other questions.


Jon

Amit Singh Mon, 01/20/2014 - 12:52
User Badges:
  • Cisco Employee,

I back-up Jon's idea. In this case you have to use IP-SLA with object tracking both on N7K and 4500-X to switch their default-routes back to Back-up site/DC or vice-versa.



Cheers,

-amit singh

Jon Marshall Mon, 01/20/2014 - 12:57
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Amit


I might be missing something here but i think you only need to track the route on the N7K. Because if the floating static on the 4500X has a higher AD the 4500X will use the EIGRP route from the primary DC.


If the IP SLA on the Nexus fails then the default is removed and so is not redistributed into EIGRP and so the 4500X no longer gets it and installs it's own default route and then advertises it into EIGRP.  If the primary link comes back up the Nexus installs it's default route into the IP routing table and advertises it and again the 4500X prefers that so removes it's own route and stops advertising.


I can't see the advantage of tracking on the 4500 because if the primary link is down it doesn't matter whether the IP SLA fails from the 4500, it has no ISP to failover to.


Am i missing something. It's been that sort of day so i wouldn't be surprised if i am


Jon

Mitchell Theriot Mon, 01/20/2014 - 13:07
User Badges:

We could point the default route at the backup site to use it's own service, that is no problem.  My concern is what if the backup circuit failed, can the backup site then failover and use the primary as the default.  This would be if for some reason something in the backup site is needing internet service for testing or some other reason.  Jon what other questions were you needing answered?

Jon Marshall Mon, 01/20/2014 - 13:13
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


You point the 4500X to the backup link with the default route but if the primary link is up it doesn't use the backup link, it uses the primary because it prefers the route it got from EIGRP than the one configured on it ie. the default route on the 4500X would have an AD > 170 and the default route it receives from the Nexus will be AD 170 so it goes via the Nexus.


So it's not so the 4500X uses the backup link even when the primary is up, it is so we can get the failover working properly.


Other questions are pretty much 1 - 3 and 5 from the above. I can check 1) if you tell me the code version but as Amit has suggested using IP SLA as well i suspect it is supported.


Jon

Jon Marshall Mon, 01/20/2014 - 13:18
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


Couple of questions about EIGRP setup. This solution requires redistributing statics into EIGRP so -


1) are you currently redistributing or using things like "default information-orginate" under EIGRP to advertise the default route into EIGRP or is it just statics you are using.


What i mean is does EIGRP advertise the default route from either switch or not ?


2) does either the N7k or the 4500X have any other statics (other than the default routes) that you would not want redistributing into EIGRP ?


Jon

Jon Marshall Mon, 01/20/2014 - 14:36
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


The interconnect. The assumption we are making is that the interconnect runs between the N7k and the 4500X and is L3 terminated on those switches either with L3 ports or SVIs.


What i mean by that is that if you tracerouted from the primary DC to the secondary DC firewall then a L3 hop would be the 4500X and vice versa. So within each DC everything is routed off their respective switch.


Apologies for all the questions, it's just that we do not want to supply you with a solution that does not work or could create problems in your network.


Jon

Amit Singh Mon, 01/20/2014 - 13:13
User Badges:
  • Cisco Employee,

Hi Jon,


I have used a lot of assumption here as we are without a network diagram. My mistake probably would have been not to read the last line carefully from the poster.


Firstly, I thought that we have FW's connected directly to both N7K and 4500 when he mentioned the ethernet connection between the DC's. I thought its a switched L2 connection not an L3 connection so we are extending vlans for L2 hearbeat between the Fw's.


Secondly, I thought that I would use floating static route without redistribution at each site both tracking the primary link using a local object in each site. When the primary goes down and my object fails, my both the core devices start pointing to a secondary/back-up firewall. That's why I mentioned using tracking on both boxes with the local objects being tracked at each site.


Its my mistake that I did not think about how would I make my floating static route into the EIGRP routing tale for others to use it as a gateway.  My bad, i should have read it properly before jumping...


See, i am getting a bit old now ...:-(


Cheers,

-amit singh

Jon Marshall Mon, 01/20/2014 - 13:24
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Amit


Can i just confirm. There may need to be a route map used with the redistribution anyway if there are other statics that shouldn't be redistributed but if there aren't i seem to remember reading somewhere that with Nexus you have to use route map with EIGRP to redistribute any statics.


Is that the case ?


Also i may very well  need your help with IP SLA  if it is different on the Nexus switches as well.


Better yet, i may just leave the entire Nexus stuff to you.


I really need to get a job where i can get my hands on these


Jon

Jon Marshall Mon, 01/20/2014 - 13:20
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Amit


Wait till you are old as me


That's one of the outstanding questions ie. does the interconnect go direct between each switch ie. it might not work if the interconnect went to other devices like firewalls first.


Jon

Amit Singh Mon, 01/20/2014 - 13:39
User Badges:
  • Cisco Employee,

Yes, that's what I thought that L2 interconnection is between the boxes directly.


Yes, you need to use the route-map for route redistribution.


http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/unicast/configuration/guide/l3_eigrp.html#wp1062253


Wait for Cisco VIRL to get started shipping. You can use NX-OS, IOS,IOS XRV and IOS-XE devices to run your topology.


http://www.cisco.com/web/solutions/netsys/CiscoLive/virl/index.html


Amit Singh

Jon Marshall Mon, 01/20/2014 - 13:54
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Amit


Scratch that question, it just doesn't make any sense.


I am not even going to look at that link yet. I have so many documents already in my list of things to read i can't afford to get sidetracked


Jon

Mitchell Theriot Tue, 01/21/2014 - 06:52
User Badges:

Thanks for the responses.  I am still gathering information on this as well.  The interconnect between the two sites does use L3 SVI's.  The plan in the future is to run OTV between the two to strech the L2 domains. A route map is already in place to redistribute certain static routes to the backup site. 

Jon Marshall Tue, 01/21/2014 - 09:20
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

Mitchell


No problem.


If you want to use the proposed solution in the meantime just let me but if you are going to be using OTV i suspect you might need a different solution. The solution i propose would work if the N7k and 4500X were peering with EIGRP and for any interconnect traffic the first device they get to is either the N7k or te 4500X.


Amit may be able to help with that as i have no experience of using OTV.


Jon

Amit Singh Tue, 01/21/2014 - 12:59
User Badges:
  • Cisco Employee,

Having OTV between the sites does not change much from the overall L3 routing toplogy. The only thing that OTV will bring in this architecture is a bit complexity :-). Not that much but your stretched L2 vlans across site will have to be designed for a proper Egress routing. Either you would end up bringing all the traffic back to main DC and then route it out to the outside world or to some HSRP filtering and let the extended vlans take an egress path from the DR site.


If you are using some L3 BGP peering on the WAN this would complicate the design as to from which site you would adverstise the IP subnet.


We will work with you when you get to that stage and i dont want to over say few thing about the future here. I would be more than happy to assit you with the design and Jon with the troubleshooting :-).


Hope this helps.


-amit singh

Jon Marshall Tue, 01/21/2014 - 13:34
User Badges:
  • Super Blue, 32500 points or more
  • Hall of Fame,

    Founding Member

  • Cisco Designated VIP,

    2017 LAN, WAN

I would be more than happy to assit you with the design and Jon with the troubleshooting :-).

Volunteering my services now

Seriously, happy to help out any way i can.


Quick question Amit. I am, funnily enough, just reading a document on OTV.


Does OTV allow you to extend some vlans and route others ?


If so how is the join interface related to the actual physical links used to connect the DCs ie.


if the join interface terminates the actual physical link then what does it do with normal routed traffic as there is no need to encapsulate it.


Jon

Amit Singh Wed, 01/22/2014 - 01:01
User Badges:
  • Cisco Employee,

Hi Jon,


Yes, OTV allows you to extend Layer 2 vlans to the remote site. That being said, OTV is typically used as an appliance model design. You stick either OTV VDC on N7K or just Cisco ASR1000 as the dedicated appliance. On N7K, the restriction is that you cannot have the OTV in the same VDC as the SVI hosted for your Vlan traffic. So you stretch your L2 trunks to OTV internal-ports and exit the extended Vlan traffic out of the "Join-Interface" which an L3 port connected back to your routed core network.


OTV join-interface encapsulates (MAC in IP) the MAC's to be forwarded to the other side and forwards it over the OTV overlay. The "join-interface" is just for the encapsulation, as long as its routed and reachable, you would be able to have a OTV tunnel established properly. In current designs on N7K, your join-interface can be a dedicated L3 routed port for Inter-DC connectivity or an L3 routed port connected back to the core router/VDC. Customer prefers the second option of having the join-interface connect back to the routed core just to save extra cost of deploying a dedicated L3 port.


The normal routed traffic is handled by the device hosting your SVI's. The traffic that needs to be routed goes stratight to your SVI/L3 gateway and exit out. The traffic that needs the L2 forwarding to the extended vlans,hits the OTV appliance internal interfaces, get encapsulated over the overlay and get to the remote side depending upon the OTV mac routes existance.


OTV can be deployed as Multicast model or Unicast model depending how the WAN core looks like related to multcast.


Hope this helps.


Cheers,

-amit singh

Actions

This Discussion

Related Content