When importing an entrust certificate, the certificate is stored incorrectly.
In particular, comparing the CA.CDS::Certificate and the CA.FED::Certificate, the IssuerDN is different.
IssuerDN = "C=US,O=Entrust\, Inc.,OU=See www.entrust.net/legal-terms,OU=(c) 2012 Entrust\, Inc. - for authorized use only,CN=Entrust Certification Authority - L1K"
*IssuerDN = "CN=Entrust Certification Authority - L1K, OU="(c) 2012 Entrust, Inc. - for authorized use only", OU=See www.entrust.net/legal-terms, O="Entrust, Inc.", C=US"
This causes transaction that leverage certificates to fail during signature validation so that the federation transaction is failing.
There is a manual workaround consisting in using XPSExplorer to modify the IssuerDN format in CA.FED::Certificate to match that in the CA.CDS::Certificat.
However, the issue is resolved in CA SSO 12.52 SP1 CR06 where a patch is included to allow for different issuerDN formats. With this fix, the right IssuerDN is picked up and Federation transactions complete without an issue. This is the recommended solution for this case
Note that the IssuerDN appears with different formats due to the presence of special characters, like backslashes, apostrophes, etc. Version 12.52 SP1 CR06 also contains fixes for allowing for assertions to be encrypted even if the IssuerDN contains non-ascii characters.