William,
You are free to limit the traffic selectors to whatever Subnet/Host/IP protocol/port you wish (both source and destination)
However bare that in mind:
- Keep the ACLs as specific as possible
- Aggregarte the ACLs whenever possible.
There is a balancing act there to be done. More access list entries will mean (potentially) more IPsec SAs with all the good and bad things which come from it.
Another way of achieving what you're looking for is using vpn-filter funcionality.
http://www.cisco.com/en/US/docs/security/asa/command-reference/v.html#wp1842564
It allows you to associate an access list which will filter inbound traffic only.
M.