TLS errors with multiple domain cluster

[mmedina77]

Observed behavior

Hi, I could use some assistance. I am setting up a cluster of 8 nodes with TLS using LetsEncrypt certificates. There are 4 separate domains with 2 systems each included within this cluster. Each system has its own individual LetsEncrypt certificate. When I use this setup, I receive errors about bad TLS certificates. When I use a single certificate with all systems listed in SANs, I do not have this problem and everything seems to work fine. I tested it originally with self-signed certificates. How can I go about setting up the cluster when there are individual certificates issued (with different domains) rather than a single certificate with SANs?

Currently, I'm connecting from the NATS CLI. Eventually will be connecting through a client API, with NATS Go client client library.

Cluster setup example:
tls://myserver1.domain.com:6222
tls://myserver2.domain.com:6222
tls://myserver3.anotherdomain.com:6222
tls://myserver4.anotherdomain.com:6222
tls://myserver5.separatedomain.com:6222
tls://myserver6.separatedomain.com:6222
tls://myserver7.lastdomain.com:6222

With individual certificates, the errors are:

• TLS handshake errors
• "Failed to verify certificate...because it doesn't contain any IP SAN's"
• Router connection closed: TLS Handshake Failure
• TLS route handshake error: tls: first record does not look like a TLS handshake

I am using LetsEncrypt certificates with full chain and root certificate included (ISRG Root X2, cross signed by ISRG Root X1)

So far, all signs point to the certificates working with the private key and CA
My only conclusion so far is that there may be an issue with NATS reading more than one entry of the certificate, or there is a requirement for SAN's to list all servers rather than individual certificates for each server.

Any help would be appreciated.

Expected behavior

Normal functionality and connectivity of the NATS cluster with TLS enabled for all modes (client-server, intercluster, etc.) using LetsEncrypt individual certificates for each system, without errors. Such as is the case with a single certificate with all systems listed in SAN's.

Server and client version

nats-server: v2.10.14

nats: v0.1.2-0.20240514111958-84fff6c55a14

Host environment

NAME="Red Hat Enterprise Linux"
VERSION="9.5 (Plow)"
Architecture: x86_64

Steps to reproduce

Create individual LetsEncrypt certificates for 4 different domains with 2 systems each.
Tested the certificates individually and also tested with all certificates combined with fullchain and root certificates included, the first certificate in the stack was the certificate for the system it was on. Errors are received.

[wallyqs]

are you using the multi cert feature?

tls {
  certs = [
    {
      cert_file: "../test/configs/certs/srva-cert.pem"
      key_file:  "../test/configs/certs/srva-key.pem"
    },
    {
      cert_file: "../test/configs/certs/srvb-cert.pem"
      key_file:  "../test/configs/certs/srvb-key.pem"
    }
  ]
}

[mmedina77]

Thank you for your reply.

No. We had cat all the certs together and used only a single TLS cert_file and key_file entry in the .conf file. Each cert needs to be listed individually for each system there instead? I would have 8 certs listed in my .conf file then, correct?

[wallyqs]

@mmedina77 yes that's correct