I have a fairly complex request. I would like to setup a high availability Internet connection and control my routing paths without accepting the whole Internet routing table.
Here is a scenario. I have two ISPs A and B. Each provides me with a class C address space. I would like to setup my ASA firewall and Internet edge routers to be able to use both providers, where ISP A is used for critical business with static NATs, and at the same time use ISP B for general usage and surfing. The configuration needs to be adaptable so that if either ISP goes down, the other takes all the traffic. So, the ASA has both static NATs with ISP A address space and dynamic NATs with ISP Bs address space. The two Internet routers are both running eBGP to their respective ISP and iBGP between them. Both receive their ISP routes and their neighbors, and a default gateway from ISP B. They are also using GLBP to form a single virtual IP for the default route from the ASA firewall, also an ISP A address. I have attempted some policy based routing with tracking routes to determine availability. It seems to be working but I am having some issues with latency and asymmetrical routing.
I have a configuration but I'm not sure if its the most efficient or if there is a better way. I have my outside interface on the ASA assigned with an address from ISP A. I guess the ASA is smart enough to do proxy ARP for any NATs that are used from ISP Bs addressing, as I only have the one outside interface configured. I have setup both ISP routers Ethernet interfaces with primary addresses from ISP A and assigned secondary IPs from ISP B. I am attempting to do some policy based routing base on source address so I can specify the egress path, for example ISP B NATs go out ISP B and static ISP A NATs go out ISP A. I also have one or two devices with ISP B addresses, outside the firewall on the segment formed by the Ethernet connections between the two routers and the firewall. The problem is that since the firewall only has an IP from ISP A, when packets are destined for these outside boxes from behind the firewall, the packets go to the routers ( the firewall default gateway) and the routers forward them off to one of the ISPs based on PBR, rather than ARPing for the machine on the local segment.
I wanted to through this out there so maybe someone will say, oh there is a much better way to do this. This seems really confusing, but maybe someone has seen this done before and has some recommendations. Thanks for any help.
The problem is that since the firewall only has an IP from ISP A, when packets are destined for these outside boxes from behind the firewall, the packets go to the routers ( the firewall default gateway) and the routers forward them off to one of the ISPs based on PBR, rather than ARPing for the machine on the local segment.
Why not simply use deny's for these destinations in your route-map acl so that it is not policy routed ?
There are however other issues with your setup i would like to expand on.
1) GLBP - gives you nothing because GLBP load-balances based on the different source mac-addresses and the ASA is the only source mac-address if the ASA is in routed mode. Mind you it doesn't hurt.
2) More importantly you want to use one ISP for all traffic if the other ISP fails. With outbound internet connectivity this is straightforward but for your static NAT statements for internal servers how will you notify clients on the internet of the new address ie. how will the DNS records be updated from ISP A addresses to ISP B addresses.
It is unlikely that each ISP will advertise the other ones class C address space so you are going to have issues. This is why in this scenario provider independant addressing is so useful ie. both ISP's advertise the same address range so you do not need to update your DNS entries for the static NATs.
Another alternative is to look at something along the lines of a GSS (Global Site Selector) which can dynamically update DNS based on availabilty of the servers but this is not trivial to design/implement.
3) Not sure why you are using secondary addressing on the ISP router interfaces.
As you say, this is not a trivial thing to do. As i say if your addressing was provider independant this sort of scenario becomes a lot easier to implement. Have a look at this doc which may give you some ideas as well -