vSphere 7.0: Understanding certificates

All communications inside vSphere are protected with TLS certificates and not the SSL. 

TLS provides secure communication for web browsers and servers. The connection is secure as symmetric cryptography is used to encrypt the data transmitted. The keys are uniquely generated for each connection and are based on a shared secret negotiated at the beginning of the session, also known as a TLS handshake.

TLS has four versions as listed below: 

• Version 1.0 
• Vulnerable and insecure 
• Not recommended 
• Version 1.1 
• Uses MD5 and SHA-1 algorithms (both considered as insecure). 
• Version 1.2 
• Uses AES cryptographic ciphers (uses SHA-256). 
• It is the current standard that is widely used. 
• Version 1.3 
• Removes weak ciphers and adds features that increase the speed of connections. 
• It is the upcoming standard. 

For more information on TLS and its versions, refer TLS. 

Supported Certificates Authorities (CA):

For vCenter Server and related machines and services, the following CA’s are supported: 

VMware Certificate Authority (Aka VMCA): 

• The VMCA acts as a root CA by default. 
• Once vCenter 7.0 is installed, it automatically creates the root certificate for signing ESXi, machine, and solution certificates. 
• Though this is an easiest solution, we need to accept a self-signed CA root certificate. 
• VMCA automates the management of certificates that are used in vSphere deployment 
• VMCA can be used as root, or as an intermediate (sub-ordinate) certificate authority. 
• VMCA issues certificates only to clients that can authenticate to vCenter SSO in the same domain. 
• VMCA does not support the following certificates. 
• Certificates with wildcards 
• The algorithms md2WithRSAEncryption 1.2.840.113549.1.1.2, 
• md5WithRSAEncryption 1.2.840.113549.1.1.4, and sha1WithRSAEncryption 1.2.840.113549.1.1.5 are not recommended. 
• The algorithm RSASSA-PSS with OID 1.2.840.113549.1.1.10 is not supported. 

Custom certificates: 

• Enterprise CA 
• Enterprise CAs are self-signed. 
• Cost effective solution. We can request as many certificates required without worrying about cost associated. 
• They provide trust mechanism on private network.
• Example: Microsoft Windows Enterprise CA 

• Third-party CA 
• Third-party CA signed certificates can be expensive and time-consuming. 
• They do provide excellent trust mechanism on public networks.
• Example: Verisign, DigiCert. 

Requirements for Certificates:

Certificate requirements depend on whether you use VMCA as an intermediate CA or you use custom certificates. Requirements are also different for machine certificates.

Do refer to VMware documentation for detailed requirements. I am not including here as this article will turn into a big chapter of some book 😅😅😅

vSphere Certificate management tools in vSphere 7.0: 

• vSphere Client (New) 
• Perform common certificate tasks web based GUI. 
• vSphere Automation API 
• Refer to VMware vSphere Automation SDKs Programming Guide. 
• Certificate Manager utility 
• Perform common certificate replacement tasks from the vCenter Server command line. 
• Certificate management CLIs 
• Perform all certificate management tasks with dir-cli, certool, and vecs-cli. 
• SSO-config utility 
• Perform Security Token Service (STS) certificate management from the command line of the vCenter Server installation. 

The following certificates are used in vSphere: 

• ESXi certificates 
• Machine SSL certificates 
• Solution user certificates 
• Internal certificates 
• vCenter Single Sign-On SSL signing certificate 
• VMware Directory Service (VMDIR) SSL certificate 

ESXi Certificates:

• ESXi certificates are stored locally on each host in the /etc/vmware/ssl directory. 
• ESXi certificates are provisioned by VMCA by default, but custom certificates can be used as well. 
• ESXi certificates are provisioned when the host is first added to vCenter Server and when the host reconnects.

Machine SSL Certificates:

• Provisioned by VMCA by default and are stored in VECS.
• Each vCenter Server node has its own machine SSL certificate. 
• The machine SSL certificate for each node is used to create an SSL socket on server side.
• All services that are running on a vCenter Server node use the machine SSL certificate to expose their SSL endpoints on SSL socket. 
• SSL clients connect to the SSL socket. 
• VMware products use standard X.509 version 3 (X.509v3) certificates to encrypt session information. 
• Session information is sent over SSL between components. The following services use the machine SSL certificate.
• The reverse proxy service:
  • SSL connections to individual vCenter services always go to the reverse proxy. 
  • Traffic does not go to the services themselves. 
• The vCenter Server service (vpxd) 
• The VMware Directory Service (vmdir) 

Solution User Certificates: 

• Provisioned by VMCA by default and are stored in VECS 
• A solution user encapsulates one or more vCenter Server services. 
• Each solution user must be authenticated to vCenter Single Sign-On. 
• Solution users use certificates to authenticate to vCenter Single Sign-On through SAML token exchange. 
• A solution user presents the certificate to vCenter Single Sign-On when it first has to authenticate, after a reboot, and after a timeout has elapsed. 
• The timeout (Holder-of-Key Timeout) can be set from the vSphere Client and defaults to 2592000 seconds (30 days). 

Internal Certificates:

vCenter SSO certificates are not stored in VMware Endpoint Certificate Store (VECS) and are not managed with certificate management tools. Generally changes are not required, but in special situations, you can replace these certificates.

• vCenter Single Sign-On Signing Certificate:

• Provisioned during installation. 
• The vCenter SSO service includes an identity provider service which issues SAML tokens that are used for authentication throughout vSphere. 
• A SAML token represents the user’s identity, and also contains group membership information. 
• When vCenter Single Sign-On issues SAML tokens, it signs each token with its signing certificate so that clients of vCenter Single Sign-On can verify that the SAML token comes from a trusted source. 
• You can replace this certificate from the CLI. However, do not change this certificate in the filesystem as it results in unpredictable behaviour. 

• VMware Directory Service SSL Certificate:

• Provisioned during installation.
• Starting with vSphere 6.5, the machine SSL certificate is used as the VMware directory certificate. 
• vSphere Virtual Machine Encryption Certificates:

• The vSphere Virtual Machine Encryption solution connects with an external Key Management Server (KMS).
• Depending on how the solution authenticates to the KMS, it might generate certificates and store them in VECS. 

VMware Endpoint Certificate Store(VECS):

• VMware Endpoint Certificate Store (VECS) serves as a local repository for certificates, private keys, and other certificate information in a trusted key store. 
• You can decide not to use VMCA as CA and signing Authority, but VECS must be used to store all vCenter certificates, keys, and so on. 
• ESXi certificates are stored locally on each ESXi and not in VECS.
• VECS runs as part of the VMware Authentication Framework Daemon (VMAFD). 
• VECS runs on every vCenter Server node, and holds the key stores that contain the certificates and keys. 
• VECS polls VMware Directory Service periodically for updates to the trusted root store. 
• vecs-cli commands can be used to explicitly manage certificates and keys in VECS. 

VECS Stores:

VECS includes the following stores.

• Machine SSL store (MACHINE_SSL_CERT)
• Used by the reverse proxy service on every vSphere node.
• Used by the VMware Directory Service (vmdir) on each vCenter Server node.

• Solution user stores
• VECS includes one store for each solution user. 
• The subject of each solution user certificate must be unique, i.e, machine certificate cannot have the same subject as the vpxd certificate.
• Solution user certificates are used for authentication with vCenter Single Sign-On. 
• vCenter Single Sign-On checks that the certificate is valid, but does not check other certificate attributes.
• The following solution user certificate stores are included in VECS:
  • machine:
    • Used by the license server and the logging service.
    • The machine solution user certificate has nothing to do with the machine SSL certificate. 
    • The machine solution user certificate is used for the SAML token exchange. 
    • The machine SSL certificate is used for secure SSL connections for a machine. 

• vpxd: 
• vCenter service daemon (vpxd) store. 
• vpxd uses the solution user certificate from this store to authenticate to vCenter Single Sign-On.

• vpxd-extension: 
  • vCenter extensions store. 
  • Includes the Auto Deploy service, inventory service, and other services that are not part of any other solution users.
• vsphere-webclient: 
  • vSphere Client store. 
  • Also includes some additional services like performance chart service. 
• wcp: 
  • vSphere with Kubernetes store.
• Trusted root store (TRUSTED_ROOTS)
  • Contains all trusted root certificates.
• vSphere Certificate Manager Utility backup store (BACKUP_STORE)
  • Used by VMCA to support certificate revert. 
  • Only the most recent state is stored as a backup, you cannot go back more than one step.
• Other stores:
  • Other stores might be added by solutions. 
  • For example, the Virtual Volumes solution adds a SMS store. 
  • Do not modify the certificates in those stores unless VMware documentation, KB or VMware Support instructs you to do so.