Originate-only would help ensure that you're the only side who can technically bring up the tunnel, however once the VPN is established the other side will be able to send traffic to your side as well until expired or torn down.
I would suggest either using a vpn-filter in the group-policy or try disabling the permit-ipsec sysopt and filter the traffic with an inbound ACL on the interface where your crypto map is applied (probably 'outside'). My concern with FTP however is knowing which ports to specifically open if passive FTP is used. Using the later method may allow the ftp inspect to dynamically permit payload traffic, but I've never tried in that scenario.
