The risks are minimal really, but I'm always hesitant to open any hole in a firewall unless you have to. Opening up ESP to any internal host is really not going to get anyone anywhere, because they first have to establish a tunnel using ISAKMP, so if you don't open that up you'll be pretty safe.
The reason you don't need to create a conduit for ISAKMP (in case you're wondering), is that ISAKMP is a UDP protocol. The PIX automatically creates conduits for TCP or UDP packets as they go out (from inside to outside), and allows those packets to come back in. ESP on the other hand, sits right on top of IP (is not TCP/UDP based),and so the PIX doesn't open a hole for that traffic to come back in, hence you have to specifically create a conduit for it.
As I said though, if you're now talking about traffic originating from the outside and coming in, you've only allowed ESP and not ISAKMP, so the risk of anyone starting a VPN tunnel to an internal host is negligible.
As I also said though, it is another hole in your firewall, so the more secure you can make it, by specifying the source and/or destination addresses, then the better that will be.
