benwaldon wrote:
Hi, I am seeking some feedback and comments in regards to the act of placing on routes hosts. This is really a not a question of what "works", but what is best practice to acheive a highly scalable, high performance, elegant solution.
The configuration is that server is connected to a switch and there are two routers connected to that switch. network 1 is behind router 1 and network 2 is behind router 2. router 1 is the server's default gateway, but the server needs to communicate to network 2.
By default, it would work if router 1 had a route to network 2 via router 2. I think that is call U-Turn Routing or Hairpin routing. It seems to me that this terribly inefficent. And if either router is actually a stateful firewall, then we might be dropping some packets also. So, I see alot of people simply add a route on the server that says reach network 2 via router 2.
However, when are talking about 20 or more servers in a high throughput enviornment, wouldn't it better to add a layer 3 switch? routers and switches are designed to make those decisions. My premise is that they can do a better job & that the host can communicate faster if it doesn't have to make routing decisions.
Is this accurate or am I just finding reasons to sell more hardware? I think it becomes more true in dynamic routing enviornment or if we use policy based routing, but it's still relevant in a basic enviornment.
Maybe I am asking the bartender if the house brew is better than import ;-). I am sure server guys would say that it doesn't matter, but I am interesting in finding out what this community thinks.
Thanks,
Ben

Ben

Personally i think adding routes to servers is generally a thing to be avoided. It's messy and it can be a sign of a badly designed network. What will happen in youe scenario is unless you disable ICMP redirects you will find that router1 will send an ICMP redirect to you server saying to get to network 2 go direct to router 2. If you disable it then yes traffic will have to go in and come out on the same interface on router1 unless you have a spare interface on router 1 and router 2 to use an interconnect.

But if you do have a high throughput environment with miultiple servers this is exactly where you should be looking at a L3 switch. L3 switches forward packets in hardware so a L3 switch will always have much greater throughput than a comparable router. So it's more to with switch vs router throughput than the server having to make it's own routing decisions.

And it's not about finding reasons for extra hardware, it's about choosing the right hardware for the job as long as you have the budget obviously.

Jon

Jon has nailed it.  This would be the best solution.

However, I have had to work with some, let's just say Cisco Nightmare solutions.

A VP would sell something with no technical backing and then me and the other engineer would have to perform the miracle.

One of the things we did was drop in 2 NIC's in the server and place them on both networks, then yes (mumble under breath) add routes in the server based on traffic.  THIS IS A BIG problem, but one that is doable, but not recommended.  I have also gone as far as 2 switches per NIC to separate traffic but also provide redundancy.  We also have taken the added step of clustering 2 servers with 2 switch connections, but that was for another solution and money was flowing.

Either way, for routing decisions you will need something to do that for you, either via switch or router.  Try to keep routing off of application devices as much as possible.