I have this problem too.
Correct Answer by Jatin Katyal about 3 years 4 months ago
When you enable Trust Communication on your primary and secondary ACS instance, and you register the secondary instance with the primary, both the primary and secondary instance check the CA and server certificates of each other. After the certificates are verified:
– If the certificates in both the primary and secondary ACS instances are valid certificates, the instances establish a secure tunnel between them and register the secondary instance to the primary.
I don't think it supports self-signed certificate however you can try installing the self-signed certificate of Primary in the secondary instance CA store and self signed certificate of secondary in the primary instance CA store.
For more information on this feature please read it here: Trust communication in distributed deployment
*Do rate helpful posts*