When using vcbMounter to take a snapshot backup of a running Exchange server the virtual machine may freeze or dismount the Information Store, particularly under heavy load.

When the virtual machine freezes, operations such as Power Off fail with the message "Operation Failed Since another task is in progress". The only way to stop the virtual machine is to kill the process:

ps aux | grep -i <Virtual Machine Name>
kill -9 <process id>

When the Information Store dismounts the following error codes are seen within seconds in the following order:

Event ID: 482
Information Store (2548) First Storage Group: An attempt to write to the file "D:\Program Files\Exchsrvr\MDBDATA\E00.log" at offset 1384960 (0x0000000000152200) for 512 (0x00000200) bytes failed after 0 seconds with system error 1784 (0x000006f8): "The supplied user buffer is not valid for the requested operation. ". The write operation will fail with error -1011 (0xfffffc0d).

Event ID: 414
Information Store (2548) First Storage Group: Unable to write to section 0 while flushing logfile D:\Program Files\Exchsrvr\MDBDATA\E00.log. Error -1011 (0xfffffc0d).

Event ID: 9558
An error occurred while writing to the database log file of storage group "First Storage Group". Attempting to unmount all databases in this storage group.

Event ID: 471
Information Store (2548) First Storage Group: Unable to rollback operation #2292176 on database D:\Program Files\Exchsrvr\MDBDATA\priv1.edb. Error: -510. All future database updates will be rejected.

Event ID: 1159
Database error 0xfffffbbe occurred in function JTAB_BASE::EcConfig while accessing the database "First Storage Group\Mailbox Store (SERVER)".

Event ID: 1115
Error 0xfffffbbe returned from closing database table, called from function JTAB_BASE::EcCloseTable on table Folders.

Event ID: 492
Information Store (2548) First Storage Group: The logfile sequence in "D:\Program Files\Exchsrvr\MDBDATA\" has been halted due to a fatal error. No further updates are possible for the databases that use this logfile sequence. Please correct the problem and restart or restore from backup.

Followed by regular messages until the services are restarted and the Information Store is remounted:

Event ID: 9175
The MAPI call 'OpenMsgStore' failed with the following error:
The attempt to log on to the Microsoft Exchange Server computer has failed.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0512-00000000

It appears that this is caused by a conflict between the Sync Driver (LGTO_Sync) included in VMware Tools to flush I/O for VMware Consolidated Backup, and the Exchange server I/O. This can be resolved by disabling the Sync Driver, or by shutting down Exchange services while the snapshot is taken.


Method 1: Online, Crash Consistent Snapshot
Disable the VCB Sync Driver (LGTO_Sync). In Device Manager:

- Select View | Show hidden devices
- Expand Non-Plug and Play Drivers
- Right click on Sync Driver and select Disable from the context menu
- Click Yes to disable the device
- Click Yes to restart the computer

As insurance the post-thaw-script.bat below can be implemented to start Exchange services in the event that the snapshot should cause the Information Store service to stop.

Alternatively don't install the Sync driver when installing VMware Tools.

Using the command line:

msiexec -i "VMware Tools.msi" ADDLOCAL=ALL REMOVE="Sync" /qn reboot=supress

Using a transform:

- Install Orca (http://theether.net/download/Microsoft/Utilities/Orca.Msi)
- Right click on the VMware Tools.msi and select Open with Orca
- Select Transform | New Transform
- Click on the Feature table in the left pane
- On the Sync feature set the level from 1 to 0
- Select Transform | Generate Transform
- Save the transform as "VMware Tools.mst"

Install VMware Tools with the following command:

msiexec /i "VMware Tools.msi" /qb TRANSFORMS="VMware Tools.mst"

Note: The same result can be achieved using the -Q 0 option for vcbMounter or the NO_QUIESCE=""; option in config.js for VCB:

"Snapshot creation failed: Custom pre-freeze script failed" when using vcbMounter (VCB without Quiesce)
http://theether.net/kb/100090


Method 2: Offline, Quiesced Exchange Snapshot
Quiesce Exchange with the pre-freeze-script and post-thaw-script below. Note this will take Exchange offline for approximately 60 seconds while the snapshot is created.

C:\Windows\pre-freeze-script.bat

@echo off
Net Stop MsExchangeSA /Y

C:\Windows\post-thaw-script.bat

@echo off
Net Start MsExchangeSA
Net Start MsExchangeIS
Net Start MsExchangeMTA

Taking Snapshots of Upgraded Virtual Machines in Nonpersistent Mode Is Not Allowed
http://kb.vmware.com/kb/2175

Virtual Machine Backup Guide
ESX Server 3.0.1 and VirtualCenter 2.0.1
http://theether.net/download/VMware/vi3_vm_backup.pdf

Ripped off by VMware: http://kb.vmware.com/kb/5962168