If understand it well, you want to enable telnet to the router on just one IP adress? This can be done easily using the following global command:
ip telnet source-interface .. (for example lo0)
This will cause that responses to telnet packets are always using this source-adress. You can then use access-lists to specify which destinations are allowed.
Regards,
Leo