Print View

How to create a self-signed certificate for VirtualCenter using OpenSSL


The default VirtualCenter certificate is only valid for 2 years from the date of install. If not using a dedicated Certificate Authority, a self signed certificate can suffice to secure the connection.

VMware have outlined the process of creating and using a root CA, rather than a self-signed certificate in the following document:

Replacing VirtualCenter Server Certificates

The instructions provided may not be complete and the following errors may be seen restarting the VirtualCenter Server service:

[YYYY-MM-DD HH:MM:SS.SSS 'App' 5944 error] [VpxKey::Decrypt] crypto failure: error:0407106B:rsa routines:RSA_padding_check_PKCS1_type_2:block type is not 02
[YYYY-MM-DD HH:MM:SS.SSS 'App' 5944 error] [VpxdCert] Failed to decrypt password: applying key to encrypted data failed (likely the wrong key)
[YYYY-MM-DD HH:MM:SS.SSS 'App' 5944 error] ODBC error: () -
[YYYY-MM-DD HH:MM:SS.SSS 'App' 5944 error] Failed to intialize VMware VirtualCenter.
Shutting down...
[YYYY-MM-DD HH:MM:SS.SSS 'App' 5944 info] Forcing shutdown of VMware VirtualCenter now

Additional errors include:

"Login failed due to a bad username or password"


The following process will create a self signed certificate valid for 10 years.

1) Create Self-Signed Certificate
1a) Generate the private key

- Run the following command:
openssl genrsa 1024 > rui.key

e is 65537 (0x10001)

1b) Generate the certificate
Ensure the Common Name below matches the DNS name of the VirtualCenter Server. Ideally use a DNS CNAME record, so the service can be moved to a different host for the duration of the certificate validity.

- Run the following command:
openssl req -new -x509 -nodes -sha1 -days 3650 -key rui.key > rui.crt

You are about to be asked to enter information that will be incorporated into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:WA
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]: The Ether
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []
Email Address []

1c) Genterate the PFX Certificate

- Run the following command:
openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx

Note: The password needs to be set to testpassword to ensure that Customization Specifications continue to work.

2) Replace the VirtualCenter Certificates

- Stop the VirtualCenter Server service with the following command:

- Rename the existing rui.key, rui.crt and rui.pfx files in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\SSL\
- Copy the rui.key, rui.crt and rui.pfx files created above to C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\SSL\

3) Update the VirtualCenter Password

- Export "HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware VirtualCenter\DB" to allow the original certificates to be reused.

Note: The password below is for the user account defined in "HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware VirtualCenter\DB\2"

- From a command prompt enter the following commands, password and verify password when prompted:
CD "C:\Program Files\VMware\Infrastructure\VirtualCenter Server"
vpxd -p

[YYYY-MM-DD HH:MM:SS.SSS 'App' 2548 info] Current working directory: C:\Program Files\VMware\Infrastructure\VirtualCenter Server
[YYYY-MM-DD HH:MM:SS.SSS 'Libs' 2548 info] HOSTINFO: Seeing Intel CPU, numCoresPerCPU 1 numThreadsPerCore 1.
[YYYY-MM-DD HH:MM:SS.SSS 'Libs' 2548 info] HOSTINFO: This machine has 1 physical CPUS, 1 total cores, and 1 logical CPUs.
[YYYY-MM-DD HH:MM:SS.SSS 'App' 2548 info] Log path: C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\Logs
[YYYY-MM-DD HH:MM:SS.SSS 'App' 2548 info] Initializing SSL
[YYYY-MM-DD HH:MM:SS.SSS 'Libs' 2548 info] Using system libcrypto, version 90709F
[YYYY-MM-DD HH:MM:SS.SSS 'App' 2548 info] Vmacore::InitSSL: doVersionCheck = true, handshakeTimeoutUs = 120000000
Enter new DB password:
[YYYY-MM-DD HH:MM:SS.SSS 'App' 2548 info] Reset DB password succeeded.

4) Start the VirtualCenter Server service

- Start the VirtualCenter Server service with the following command:

5) Reconnect ESX hosts
The ESX hosts need to be disconnected/reconnected to VirtualCenter. If this is not done the vim.fault.InvalidLogon dialogue may appear.

From the VMware Infrastructure Client perform the following for each ESX host:

- If ESX host is not disconnected right click on the ESX host and select Disconnect from the context menu
- Right click on the ESX host and select Connect from the context menu
- Enter the Username and Password and click Next
- If the "Duplicate Management: The host is already being managed by IP Address" warning appears click Yes
- Click Next to accept the summary
- Select the folder for the virtual machines and click Next
- Click Finish

The certificate will need to be imported as a Trusted Root Certificate Authority to be fully trusted and pass the following three test:

- The security certificate is from a trusted certifying authority
- The security certificate date is valid
- The name on the security certificate matches the name of the site

This can be done from the web browser, the CERTMGR.MSC console, or the MMC using Add/Remove Snap-ins | Certificates.


VirtualCenter Server Fails to Start After You Replace Default SSL Certificates with Custom SSL Certificates

Developer’s Setup Guide VMware Infrastructure SDK 2.5

To modify the Web proxy service on VirtualCenter Server 2.5 to support HTTP:

- Log in to the VirtualCenter Server system as the Windows Administrator of the machine.
- Change to the directory containing the proxy.xml file: c:\Documents and Settings\AllUsers\Application Data\VMware VirtualCenter
- Use a text editor to open the proxy.xml file.
- Find the section of the file associated with the /sdk.
- Change the accessMode to httpAndHttps.
- Restart the VirtualCenter Service from a command line or from the Windows Services control panel.

Replacing or Regenerating an SSL Certificate for the Management Interface


VMware VirtualCenter 2.5
VMware vCenter Server 4.0 Update 1

Created: 16th December 2008
Updated: 14th April 2010

Print View

© 2005-2024 Jamie Morrison