I used the "originate-only" and it works just fine. But recently I have implemented another setup where instead of exposing the segments of interest on the inside to the other side I PAT to a private network thereby hiding the segments from the other side. So not only they cannot send any traffic but also there have no visibility or expose to my internal network.

Not sure if that concept can be applied on a "two way" tunnel but rather only on a tunnel that traffic is one way.

Here's what I mean:

interesting traffic --> PAT (private IP) --> cryptomap & nonats of the PAT'd address --> Internet --> other side of the tunnel.

So again, the other side of the tunnel does not know anything about the interesting traffic.