Hi there,
Well VPN clients cannot connect if you dont configure your PIX for them. So you dont have configure dynmaic crypto maps etc, do all LAN-to-LAN IPSec in which case VPN clients cannot connect. If you want to restrict some users, then you can probably look at some access-list controls for the IPs which will connect.