I dont think there is a max in CallHandlers, but you are limited in the number of Ports you can use for CallHandlers. If you have 96 ports being used, the 97th caller will get a busy signal.
In really large enviroments, it may be advisable to configure a seperate Unity server just for CallHandlers if you are trying to route many things from incoming remote sites. (hub and spoke designs)