ACE Routed Mode vs One Arm Mode

patelvc7601 · ‎02-13-2012

We have Currenlty Deploying the ACE in Our Enviornment. After Reading One-armed Mode & Routed Mode I have come with Pros & Cons of both approach & I wanted to make sure my understanding is correct . I also wanted to find how others are deploying int their Enviornment .

1 One Ared mode : Routed Mode

Pros: Pros:

1 Allows to put Load Balanced Server Anywhere or 1 Direct Access to the Server is Not possible.

2 Ability to Bypass ACE for troubleshooting ( Security Concern) 2 No Source NAT Necessary for the Client.

Cons: Cons.

1 Direct Access to Server IP is possible . 1 Even non-load balaced traffic is passing through ACE

2 Client IP is NATED which creates Security Issue if logging are done by IP .

I know this list is Partial & somewhat which perspective you are looking.Please Share your experience . While deploying ANM 5.1 there was brief mention about NAT makes impractial in Some situation with One-Armed Mode but did not give any example.

alois.heilmaier · ‎02-13-2012

Hi, I do have 2 examples when to not deploy SNAT:

a) It's a self developed Application. It will deploy Software to the clients. The Deployment process needs the source IPs of the client to work. Unfortunately I do noch know more about that application but that's a reason to not deploy SNAT.

b) SMTP security policy based on source IP. E.g. on an Exchange hub transport server you can ACL allowed clients based on source IP

So if you want best of both worlds you need to deploy both. You can deply both worlds with two armed mode.

On the loadbalancer you have a frontend network where the VIP's are placed and a backend network, where the realservers are placed.

With SNAT you can have a routed netwok between Loadbalancer and realserver or in case realserver network is directly connected to loadbalancer, just use the router for default gateway on realservers. So direct access to real servers gets routed via router, loadbalanced access to servers gets loadbalanced with client SNAT.

Without SNAT you have to connect the realserver network directly to the loadbalancer, use no router for default gateway instead use the loadbalancer as default gateway. But If you want direct realserver (e.g. for Disk-Backup) access you have to use a router and also routing entries on the realservers and this can be troublesome when not deployed carefully.

patelvc7601 · ‎02-20-2012

Thank you all for sharing your experience.

jburmania · ‎02-13-2012

We use the routed mode for all of our load balanced environments; however, the ACE is not the default gateway for the backend servers (real servers). The ACE has an interface on the backend server network, there is also a SVI (switch) within the backend subnet for the default gateway of the backend servers. With this setup we are able to setup multicast on the backend subnet (via the SVI) and traffic that needs to talk directly to the server (backup traffic, etc...) can without having to go thru the ACE.

Because the ACE isn't the DFG we have to source NAT to make sure that load balanced traffic comes back to the ACE from the backend servers... this does create issues when the application needs to know the clients IP address. However, we have used the ACE's "Client IP Header Insert" functionality to mitigate this...

http://www.cisco.com/en/US/products/hw/modules/ps2706/products_configuration_example09186a00809c3041.shtml

We have a lot of load balanced environments with many VIPs and we have had success using the routed mode.

Borys Berlog · ‎02-17-2012

Hi. Replying here as not a Cisco employee but from previous experience :).

At my previous job we used ACEs firstly in one-Arm mode and had a lot of organization issues. The main problem was that in this case routing table on servers should be carefully kept up to date and all changes has to be properly implemented on network and server side. In our case we had a lot of servers (hundreds) and server team/ application team and network team were 3 separate independet teams. It was hard to managed and we had a lot of issues when something was changed by one of the teams and then we got asymmetric routing.

We found 2 general solutions:

- use SNAT (the main drawback here that a lot of application had to have access to real client IP and NAT wasn't acceptable.)

- use Routed mode (drawback of this mode you listed in your first message, however our servers had a few interfaces and connection to some backbone stuff like : backups, DB servers , etc - was done though different interface and not via ACE. In this case it's much easier to maintain routing table on server, as you have a couple of routes to set of backup servers (their IPs usually don't change often) and default route to the ACE)

Of course you can combine these two modes on one ACE and use appropriate mode for each serverfarm, however in this case you should check if such complication really makes sense. As in case of a big/global spread network team it can be an important point and traffic load on ACE may be not so high.

Surya ARBY · ‎02-20-2012

Another experience from me

For non web application one arm can be a nightmare

For web applications one arm is definitely the scenario I prefer :

- using source nat is my best practice, you can put your LB on the production and make your tests without impacting the whole network topology, the only caveat appears when the application needs to log the IP address but to me I have never found it to be an issue as IP insertion in the header is the standard method for years now.

For development, catching the IP address is usually a matter of minutes to modify the code.

sabareeswar · ‎02-25-2012

Hi Surya,

Could you pls elaborate your quote "for non web applications, one arm mode can be a nightmare" ?

Surya ARBY · ‎02-26-2012

If you use source nat you'll hide the source IP address of the requests, it can be an issue.

If you need to check source ip, you'll have to deploy PBR, which is cannot be deployed everywhere and is not easy to operate.