If you just want to authenticate the users of this group to the Radius server, leave the Group set to Internal, go under the IPSec tab and set Authentication to Radius.
Setting the group to External means you want to configure the entire group parameters on the Radius server (you'll notice the configuration tabs disappear from the VPN3000 GUI). A Win2K Radius server doesn't have the ability to set all the different attributes for the group which is why it's failing.
Setting it to External works with a ACS server cause it has all the definable attributes, but even then not many people use it (I don't see any point in it personally). In general the wording on the screen just confuses people and they do what you've done.