Sure you can. I've done something like that a couple of different ways:
1. Using ssh and X11 forwarding (not IPSec, but close)
2. Using something like VNC (or its variants) over an IPSec tunnel.
In the first case, I ssh'd to my linux box at home (on Roadrunner) and started a display on my work laptop running Cygwin. It works reasonable well. Startup seems to be pretty slow though.
In the second case, I've VPN'd to work and used VNC for managing some Windows boxes. There are VNC clients for unix boxes and this will generally be quicker than doing an actual X display. I haven't played around with the unix version of it - I don't know if it's one VNC session per physical display or per user or what.
So, I don't think latency is really the issue, it's more bandwidth since there's lots of data that has to be drawn, re-drawn with things like mouse movements, windows opening/closing, etc.
-Eric
