Direct connect server to server is a good benchmark, especially if the servers are identical with identical NICS. A couple of things to check would be the negotiated speed and duplex of the servers connections to your switch. Make sure that they are connecting at the optimal speed. If they aren't negotiating properly, as sometimes happens, try manually setting your speed and duplex settings until you get the results you want. I've seen this a couple of times with Gigabit interface cards on servers. Another thing that I've seen help is to enable portfaast on the ports that your servers are plugged into.
set spantree portfast 3/1 enable (where 3/1 is your port number)
This really shouldn't make any difference with port speed negotiations, but I've seen cases where it took care of the problem. Just be sure the ports you enable portfast on are connected to servers or workstations as the blocking, listening and learning states of STP are skipped with this command.
Beyond that, make sure your drivers and BIOS versions are current on your servers. Servers frequently ship from the vendor with outdated drivers loaded and new installs should always be updated.
Bob
