I think you are correct in pointing that out. How many clients are you expecting at the site? have to tried roaming on a open ssid? i would suggest you do that to see if there is some other issue than time taken for authentication.
Also, if the client count is going to be less than 100 and all of them are going to use wpa2/psk, you can try using local authentication whihc may reduce auth time.