This should work, although you may want to test it to ensure it acts exactly as you wish.
You need to configure a dialer map for every user that you want to connect; ie
dialer map ip 1.1.1.1 5551212
When 5551212 dial in, they will be allowed to connect and given an address of 1.1.1.1. Any CLID that dial in but does not match a dialer map should be disconnected.
The only other option that I can see it try try pre-authentication, but that would require an AAA server.