Many thanks for the information Jaimie

cheers,

Azeez

Don't hold your breath, it is very unlikely it will ever be supported as wildcard certs are not considered very secured.

Cheers Chris, will try not to for too long
 

:-)

SAN certs are the way to go where you can cover all your CUCM and IMP nodes for example with single Tomcat SAN cert. And another SAN cert for XMPP, etc.

What if you have a wildcard that supports SANs? Has anybody tried one of those successfully using the Multi-server certs features in 10.5?

I'm interested in buying the fewest certs possible for a BE7000 cluster including 3 CUCMs, 2 UCX, and 2 IMP nodes and would want to cover Tomcat, Call Manager, and XMPP.

I see that under the multi-server CSR dialog that my CUCM and IMP servers are all auto-populated but Unity is not. Unity appears to do a similar but separate thing with it's own servers.

So, does anyone know if any of these servers/services will accept and correctly function with a cert that has a wildcard Common Name and correctly populated SANs?

Unfortunately the products expose no way to import a certificate AND private key pair, so Cisco gives us no supported mechanism to import your (hypothetical) wildcard certificate.  They only allow you to generate a CSR, and import the resulting certificate.  

It is the hope of many that Cisco one day rectifies this and allows us to import certificate/private key pairs (as on nearly all other platforms that use certificates, Cisco included).

-jd

Yes, but with a Digicert wildcard you can pay once and use a CSR from each server to generate unlimited duplicates of that wildcard that are individually keyed with unique SANs per dupe. No private key import since each dupe is individually keyed. If a host's key is compromised you don't have to replace all deployed copies. It's quite an awesome offering.

I have since found a gossamer-threads thread with multiple folks claiming they have it working for multiple clustered services on 10.5. I intend to try it out soon.

Your response has piqued my interest.  Could you share link(s) to the gossamer-threads thread you're referring to?

I bought a Digicert Wildcard Certificate ($595/yr) just now but I am unsuccessful at loading the resulting certificate in to CUCM 10.5.2.13036.  The error it outputs is complaining about the CSR's SAN field and the resulting certificate's SAN field being different.

I can't find a combination wherein the SAN fields will line up:

* If I generate the tomcat CSR on CUCM as a single server certificate, the SAN field in the CSR contains DNS:$cucmfqdn (non-configurable), and

* If I generate the tomcat CSR on CUCM as a multiserver certificate, the SAN field in the CSR will contain DNS:$eachclustermemberfqdn (non-configurable) 

So in neither case do the SAN fields line up, so CUCM will not load the resulting certificate.  There must be some special sauce I'm missing on the Digicert side of things.

-jd

I'm on mobile at the moment but my recollection from the thread/s is that when you import the CSR to Digicert you need to manually input the SANs in the available fields. They may be hidden in an expandable field.

I'll attach the links later.

Yes, that appears to be the case.  When you generate a "duplicate" (Digicert's term for additional certificates generated from new CSRs) you are presented with the ability to populate up to 10 SAN fields with $hostname.

I was able to generate a multi-server tomcat certificate for a small CUCM 11.0(1) (one CUCM node, one CUCM IM&P) cluster and load the resulting Digicert certificate.  The CN doesn't match but CUCM doesn't seem to care as long as the SAN fields line up.

I'm going to try using the same Digitcert wildcard "duplicate certificate" process for cup and cup-xmpp, will report back with my findings in a few hours.

This is an interesting find and further proof that Digicert is the best CA option out there.  Love those guys.

Ok, so, kudos to you because you're definitely on to something here. I spent some time today generating CSRs and feeding them in to the Digicert "duplicate" wildcard certificate process for a pretty healthy number of servers (replacing certificates signed by a private CA):
* 2-node CUCM SME 10.5.2.13036, Tomcat (multiserver)
* 2-node CUCM 10.5.2.13036 & 2-node CUPS 10.5.2.22900, tomcat (multiserver)
* 2-node CUCM 10.5.2.13036 & 2-node CUPS 10.5.2.22900, tomcat (multiserver)
* 1-node CUCM 11.0.1.10000 & 1-node CUPS 11.0.1.10000, tomcat (multiserver)
* 2-node CUPS 10.5.2.22900, cup (individual)
* 2-node CUPS 10.5.2.22900, cup-xmpp (multiserver)
* 2-node CUPS 10.5.2.22900, cup (individual)
* 2-node CUPS 10.5.2.22900, cup-xmpp (multiserver)
* 1-node CUPS 11.0.1.10000, cup (individual)
* 1-node CUPS 11.0.1.10000, cup-xmpp (individual)
* 1-node Connection 10.5.2.13036, tomcat (individual)
* 2-node Connection 10.5.2.13036, tomcat (multiserver)
* 1-node Connection 11.0.0.99833, tomcat (individual) [hmm, looks like I forgot to upgrade this guy after 11.0 was released]

Two issues I ran in to:
* Any/all SAN entries in the CSR must also be found in the resulting certificate. Some servers apparently had an alternate name defined previously (set web-security unit org city state country [altname]) and the resulting certificate would fail to import since I didn't think/know to pass that name along to Digicert. I opted to eliminate the alternate name by running "set web-security" with no altname. This triggered a fallback to self-signed tomcat certificate, I would then generate a new CSR and load it up without issue. Also you can use openssl to view and compare the values in the SAN field both the CSR (openssl req -in $csrfilename -text) and the certificate (openssl x509 -in $certfilename -text).
* I ran in to CSCup59247 on one of my clusters for the third time in my life -- OS Administration shows zero traces of the old cup-xmpp certificate, but one of my CUPS nodes would keep serving the old one up even after reboots, service restarts, etc. as the bug ID's workaround section suggests. I figured out I could root in to the node that was serving up the correct certificate and simply copy it (/usr/local/xcp/certs/xmpp.pem) to the node serving up the old certificate, and then restart the "Cisco XCP Router" service to make it take effect. No idea why this bug is terminated as it's obviously real and out there.

I tested a ton of things that interact with all of these certificates:
* Tested Jabber logins from various client (Windows, Android). Of special note, once all the correct certificates were in place I was able to perform a first-time Jabber for Android login without getting a single certificate warning which wouldn't have been possible with the previous private CA-signed certificates I had in place without manually loading the CA cert on the mobile device -- that was rendered unnecessary. I have a number of deployments in the real world that can do this as well, but only by spending a significant amount more than $595/year to purchase all of the various certificates.
* Tested intercluster IM between the three CUPS clusters. Tested IM to external/XMPP-federated (via Expressway) contacts. Tested group chat on all CUPS clusters, with a mix of users joining from each CUPS cluster.
* Tested bringing up the service list of other servers from CUCM Servicibility,
* Tested TFTP Proxy and confirmed it to still be operational,
* Tested to ensure ILS and GPDR replication was still taking place across all of the CUCM clusters

Three concerns:
1. Cisco's favorite answer to anything is "that's not supported", so I fear running in to an odd corner case only to have Cisco say "sorry, what you're doing with certificates it not supported", and
2. I'm concerned about the limitation that the 10 names in the SAN field, they must be within the same domain as the domain wildcard itself. For example, if your domain wildcard certificate is *.domain.com, your 10 subdomains in the SAN field MUST be $blah.domain.com. In many (most?) environments this is likely fine, but I can foresee trouble in hosted/managed environments where the infrastructure might use one domain while the clients might use another. There's no easy mechanism to, for example, allow a cluster to point a friendly name (ucmuser.customerdomain.com) since customerdomain.com isn't domain.com.

I'll probably beat on this some more next week as I think this is a very compelling solution worth pursuing further, and thank you for pointing out that Digicert does this with their wildcard certificates.

-jd

Wow, all I can say is thank YOU for bothering with all of that validation and laying out your findings so completely and concisely. It's comforting to know that with our small simple clusters we should be relatively safe operationally  barring an issue where TAC dings us on the unsupported use.

You covered several checks I wouldn't have known/thought of so once again, thanks.

So I don't have a test cluster to play around with but I wanted to run this by you before I go generating CSRs since you appear to know these components inside and out.

I have a BE7000 cluster at 10.5.2.22900-2 with:

3x CUCM

2x CUPS

2 x Connection

I was planning to do separate duplicates for the following certs:

CUCM -

tomcat(multiserver)

Callmanager(multiserver)

CUPS -

tomcat(covered by CUCM multiserver above?)

cup(individual per server with FQDN in SAN)

cup-xmpp(multiserver)

cup-xmpp-s2s(multiserver)

Connection -

tomcat(multiserver)

After installing issued certs and CA trust certs  I would just restart the related services.

Do you see anything missing from or unnecessary in this list?