2 or 3 access layer switches in every server rack? Considering typical stand-alone rack switches offer 24 or 48 ports, how many servers per rack or how big are these racks?! Or, were your intended access layer switches only 4 port switches?! (laugh)
The main disadvantage of rack switches, you either run out of ports or have too many ports. Since server hosts often push bandwidth harder than user hosts, you also have the issue of how much aggregate bandwidth you need to move out of an individual rack. These issues are better addressed by your suggestion of using patch panels from each rack that come to a network switch rack.
You're correct, all the cables, and their organization, has an expense, but this has to be considered against switch costs. Assuming you intend to use Enterprise switches that offer gig to the servers and perhaps 10 gig uplinks, there's a cost to having extra swiches by deploying to each rack. Also switches per rack would disallow the advantages you might find with a chassis switch in a switch rack. Further, depending on performance to provide, do you want to be limited by uplink bandwidth from each rack, or pay for high bandwidth from each rack that's usually underutilized.
Besides "looking right", the foregoing is often why there are server rack patch panels, etc.
One design, that I found interesting, was using the 3750 "stacked" across racks (guess you could do same with newer 2975s).
