I had this problem and fixed it. We were absolutely positive that the FQDN configuration on the OpenAM server was correct. Certainly verify that first on your server, but it's not the ONLY thing that throws this error.
Looking at the OpenAM debug logs set at the Message level in the Authentication file, I was able to see numerous failed authentication attempts for the "demo" user when I tried to enable SSO.
We had earlier removed the demo user because it shouldn't be really needed for any production OpenAM deployments.
We were wrong.
I added the demo user back to the OpenDJ embedded database "Access Control > Top Level Realm > Subjects" and then I was able to enable SSO on my CUCM server.
The UserID is "demo" the password is "changeit" and all fields are mandatory, even thought First Name doesn't always have the * that indicates it is mandatory.
Hopefully there will be either a documentation defect or a code defect coming out of this recent discovery.
These are the paths to get to each CCX logs through CLI. They may be helpful if you are having issues accessing RTMT or downloading logs through it.
If you want to download them you have to prefix "file get " and you can add one of the options (re...