When running the VMware-vcserver.exe the process will try to launch the MSI installer and bring up the Windows Installer help due to an incorrect command line.

Process Monitor from SysInternals reveals the command line being used by VMware-vcserver.exe to launch msiexec:

MSIEXEC.EXE /i "C:\Temp\VI2.5.U3\vpx\VMware VirtualCenter Server.msi" /L*v C:\DOCUME~1\username.DOMAIN\Local Settings\Temp\1\vim-vcs-msi.log  REINSTALL=ALL REINSTALLMODE=vomus  IS_MINOR_UPGRADE=1 SETUPEXEDIR="C:\Temp\VI2.5.0U3\vpx"

From this it can be seen that the %TEMP% variable has a space in the variable, and is not the typical 8.3 filename.

From a command prompt:

SET TEMP=C:\TEMP
VMware-vcserver.exe

The MSI should now launch successfully.