Latency for the IPSec tunnel should not matter much. It effects the traffic that is flowing through the tunnel. You need to make sure that the path between the 2 peers is not dropping any packets as that will cause you to loose DPD (keepalive) packets, that will take the tunnel down from time to time.