Hi, I do have 2 examples when to not deploy SNAT:
a) It's a self developed Application. It will deploy Software to the clients. The Deployment process needs the source IPs of the client to work. Unfortunately I do noch know more about that application but that's a reason to not deploy SNAT.
b) SMTP security policy based on source IP. E.g. on an Exchange hub transport server you can ACL allowed clients based on source IP
So if you want best of both worlds you need to deploy both. You can deply both worlds with two armed mode.
On the loadbalancer you have a frontend network where the VIP's are placed and a backend network, where the realservers are placed.
With SNAT you can have a routed netwok between Loadbalancer and realserver or in case realserver network is directly connected to loadbalancer, just use the router for default gateway on realservers. So direct access to real servers gets routed via router, loadbalanced access to servers gets loadbalanced with client SNAT.
Without SNAT you have to connect the realserver network directly to the loadbalancer, use no router for default gateway instead use the loadbalancer as default gateway. But If you want direct realserver (e.g. for Disk-Backup) access you have to use a router and also routing entries on the realservers and this can be troublesome when not deployed carefully.
We use the routed mode for all of our load balanced environments; however, the ACE is not the default gateway for the backend servers (real servers). The ACE has an interface on the backend server network, there is also a SVI (switch) within the backend subnet for the default gateway of the backend servers. With this setup we are able to setup multicast on the backend subnet (via the SVI) and traffic that needs to talk directly to the server (backup traffic, etc...) can without having to go thru the ACE.
Because the ACE isn't the DFG we have to source NAT to make sure that load balanced traffic comes back to the ACE from the backend servers... this does create issues when the application needs to know the clients IP address. However, we have used the ACE's "Client IP Header Insert" functionality to mitigate this...
We have a lot of load balanced environments with many VIPs and we have had success using the routed mode.
Hi. Replying here as not a Cisco employee but from previous experience :).
At my previous job we used ACEs firstly in one-Arm mode and had a lot of organization issues. The main problem was that in this case routing table on servers should be carefully kept up to date and all changes has to be properly implemented on network and server side. In our case we had a lot of servers (hundreds) and server team/ application team and network team were 3 separate independet teams. It was hard to managed and we had a lot of issues when something was changed by one of the teams and then we got asymmetric routing.
We found 2 general solutions:
- use SNAT (the main drawback here that a lot of application had to have access to real client IP and NAT wasn't acceptable.)
- use Routed mode (drawback of this mode you listed in your first message, however our servers had a few interfaces and connection to some backbone stuff like : backups, DB servers , etc - was done though different interface and not via ACE. In this case it's much easier to maintain routing table on server, as you have a couple of routes to set of backup servers (their IPs usually don't change often) and default route to the ACE)
Of course you can combine these two modes on one ACE and use appropriate mode for each serverfarm, however in this case you should check if such complication really makes sense. As in case of a big/global spread network team it can be an important point and traffic load on ACE may be not so high.
Another experience from me
For non web application one arm can be a nightmare
For web applications one arm is definitely the scenario I prefer :
- using source nat is my best practice, you can put your LB on the production and make your tests without impacting the whole network topology, the only caveat appears when the application needs to log the IP address but to me I have never found it to be an issue as IP insertion in the header is the standard method for years now.
For development, catching the IP address is usually a matter of minutes to modify the code.
If you use source nat you'll hide the source IP address of the requests, it can be an issue.
If you need to check source ip, you'll have to deploy PBR, which is cannot be deployed everywhere and is not easy to operate.