I've been banging my brain against this for a bit, and I need some advice from those more experienced.
The diagram I've attached crudely shows how our WAN is currently set up (sorry for my poor diagram skills).
Site A, Site B, and Site C communicate with each other over VPNs on fiber Internet connections.
Site D communicates with Site A over a point-to-point T1, and to all other sites over VPNs on a DSL Internet connection.
Site E communicates with all sites over VPNs on a DSL Internet connection.
Site A, Site B, and Site C have the ability to fail-over to VPNs over DSL if their Fiber connection is down.
Site D can communicate with Site A over VPN over DSL if the T1 goes down.
The benefits of this setup are:
1. Failover is automatic using route tracking at Site A, Site B, and Site C. If the Fiber at Site B goes down, The ASA changes its default route to the DSL. Access-list rules are in place to allow traffic through VPN over either the Fiber interface or the DSL interface. All other ASAs have both the Fiber and DSL IPs as peers, so they can communicate to either.
Likewise, at Site D, if the router senses that the T1 is down, it falls back to a route sending traffic bound for Site A to the ASA5505.
(Obviously, Site E has no failover option.)
2. Sites communicate directly with each other - Site B has a VPN to Site A, Site C, Site D, and Site E. No routes pass through another Site to reach their destination.
However, communication with Site A is by far the most important factor. In a failover situation, we are able to sacrifice communication between Site D and Site B, for instance.
We are looking at moving to a carrier ethernet solution. Basically this just gives us Layer 2 connectivity between some of the sites. This would replace the Fiber connections in the current diagram. We would, however, retain the Fiber connection to the Internet at Site A. We would also retain the DSL lines at the sites that currently have it, for failover. Site E would remain DSL only, Site D would remain on a T1 (with the DSL for failover).
We are trying to accomplish the following if we move to the carrier ethernet solution:
1. Route all Internet traffic (with the possible exception of Site E) through the Internet connection at Site A.
2. Continue to encrypt internal traffic, even coming over the carrier ethernet.
3. Failover to VPNs over DSL if the carrier ethernet goes down automatically. If not automatically, with as little manual intervention as possible.
So, the big question is, can I do this with our existing equipment? And I don't think I can, if for no other reason than I'm going to be out of interfaces on the ASA5510 (in addition to what's on the diagram, there's a DMZ port in use). So, what if I add another ASA at Site A, specifically for Internet traffic (and the VPN to Site E)?
Let me start with the first thing we are trying to accomplish. Can I route all the Internet traffic through Site A, assuming the carrier ethernet is up and working at all sites? If so, how? Should I encrypt all traffic, send it through a VPN, nice and NAT-exempt, and then route it out a second ASA at Site A?
I realize this is pretty long and involved. I would appreciate any advice anyone has. Thanks!
I caution that a sales engineer ought to be consulted for the proper platform. That said I would think a 1800 or 1900 series would be good for your branches. Your hub site may be a 2800/3800 or 2900/3900 series router depending on throughput there; I would imagine it would be greater than 10Mbps? These routers support a VPN accelerator card that offloads the encryption process into hardware vs. software. A sales engineer will be well versed in all these options.
One critical descsion point to to make sure you get a feature set that supports dynamic routing, GRE and IPSEC. The Cisco sofware advisor will be helpful here.
Since you already have EIGRP in your network and your engineers are already already familiar with this protocol I'd stick with it; make this an EIGRP network. Utilizing GRE will create tunnel interfaces; you'll be able to adjust the delay attribute on these interfaces to cause one to be preferred over the other. These routers can NAT or not based on policy and egress interface. The encryption decision will be based on whether or not a GRE tunnel interfaces is selected for packet forwarding. The interface selection decision will be based on route metrics in EIGRP.
You are correct; the traffic forwarding decision will be unaware of the underlying media of internet or carrier Ethernet. It will only be aware of the tunnel interface it forwards over and the delay attribute you select for the tunnel interface. You will need to adjust the MTU but that's another conversation.
I would think you'd have two tunnels built and ready for use at all times. One over carrier Ethernet, the other over the internet. Only the more attractive tunnel would forward traffic while the other waits for a failure situation.