Exchange Server 2010 virtualize or not? That’s the question

<update March 2012> Microsoft does not support running Exchange databases or transport queues on NFS storage, even when the NFS storage is presented to the Exchange VM as block level storage

—-

In Exchange Server 2010 Microsoft introduced the Database Availability Group or DAG for short. A group of up to 16 Exchange 2010 Mailbox servers can be part of a DAG. A DAG offers redudancy at the database level for hardware, network and database failures. Basically, a mailbox store is made redundant by placing the data in at least two mailbox database located at physically different disksets.

As Exchange 201o does not require a lot of disk resources (IOPS, even a couple of low cost harddisks (Just a bunch of disks) JBOD wil in most cases be sufficient. Microsoft recommends to install Exchange Server 2010 on a physical server, with local storage (either inside the server, or if more capacity is needed, as direct attached storage).

But what about virtualization? Just as more and more organizations are recognizing the benefits of server virtualization (costs, power, disaster recovey etc) and even the most demanding applications are ready for virtualization, Microsoft tells to use local storage instead of shared storage and to use physical hardware.

What should be considered when deciding using physical hardware to use for your Exchange 2010 mailbox server which is member of a DAG:

-server hardware costs
-remote management cost (HP iLO for example needs a paid license)
-space in your serverroom or datacenter
-cooling costs
-power costs
-hardware maintenance costs
-Windows Server license costs
-installation costs (someone needs to install the server in a rack, plug it on the network, storage, install the operating system  etc)
-scalability. If more diskstorage is needed, can strorage easily be expanded without addional expenses for extra storage cabinets?
-security of your mail if the server gets stolen or if data is accidently not removed after phasing out of server

For Exchange 2010 running as a virtual machine you still have hardware costs. Lets say 15 VM’s can run on one host. The cost for server hardware is much less then for a phyiscal server. Same for power, cooling, maintenance, etc. If you buy a Windows Server Datacenter Edition, the license for your VM is free. The big difference in costs is the cost for shared storage.

Does Microsoft support running Exchange 2010 mailbox servers part of a DAG as a virtual machine?

Yes, they do, but under certain conditions:
1. you need to disable live migration of virtual machines
2. you need to disable automated re-start of VM’s if a host has failed
3. Microsoft does not support if Exchange data is stored on NAS devices. So storing an Exchange mailboxstore on a VMware NFS volume is not supported. It will work perfectly but Microsoft has not tested this scenario and thus does not support it.

See the Exchange 2010 system requirements . Under hardware virtualization it is stated:

 

Clustered root servers is another term for VMware ESX (Hyper-V ) hosts which are part of a VMware HA (Microsoft Fail Over cluster) enabled cluster .

To have Microsoft support your Exchange 2010 mailbox server virtual machine you should disable HA and disable DRS.

Microsoft states NFS or other NAS devices type of presenting data is not supported for Exchange 2010 in this article. The same is true of Exhange 2007.

The article states:

The storage used by the Exchange guest machine for storage of Exchange data (for example, mailbox databases or Hub transport queues) can be virtual storage of a fixed size (for example, fixed virtual hard disks (VHDs) in a Hyper-V environment), SCSI pass-through storage, or Internet SCSI (iSCSI) storage. Pass-through storage is storage that’s configured at the host level and dedicated to one guest machine. All storage used by an Exchange guest machine for storage of Exchange data must be block-level storage because Exchange 2010 doesn’t support the use of network attached storage (NAS) volumes. Also, NAS storage that’s presented to the guest as block-level storage via the hypervisor isn’t supported. The following virtual disk requirements apply for volumes used to store Exchange data:

vSphere 4 Update 1 offers new functionality in which HA and DRS can be disabled for an individual virtual machine, instead of for the whole ESX/ESXi host. This would mean you can run Exchange 2010 mailbox servers virtual machine on a VMware host and comply to the Microsoft condition. The Exchange vm will not be restarted in case of a host failure. And the VM will never be Vmotioned to another host.

How to configure a Exchange 2010 virtual machine running on ESX 4.x to comply to the Microsoft support policy?

A couple of steps needs to be taken on a ESX 4.x VMware HA\DRS cluster. Mind, when running ESX 4 you need Update 1 to be able to exclude a single VM from restart after a host failure.

1- create an anti-affinity rule to make sure the Exchange 2010 DAG servers are never run on the same ESX host. If the host fails, your DAG is gone.

2- disable DRS for the Exchange 2010 VM

3-disableVirtual Machine restart for the Exchange 2010 when the ESX hosts fails

Create anti-affinity rule of all Exchange 2010 mailbox servers

1 In the vSphere Client, display the cluster in the inventory.
2 Right-click the cluster and select Edit Settings.
3 In the left pane of the Cluster Settings dialog box under VMware DRS, select Rules.
4 Click Add.
5 In the Rule dialog box, type a name for the rule. (something like ‘separate Exchange MBX servers’)
6 From the Type drop-down menu, select a rule.
-select Separate Virtual Machines.
7 Click Add and select the two Exchange 2010 virtual machines to which the rule applies, and click OK. If you hav more than two Exchange 2010 mailbox servers, you should create another two anti-affinity rules to keep server 1 and 2 separated, server 1 and 3 separated and server 2 and 3 separated.
8 Click OK to save the rule and close the dialog box.

Enable Strict Enforcement of Affinity Rules
To ensure that affinity rules are strictly applied, set an advanced option for VMware DRS. Setting the advanced option ForceAffinePoweron to 1 will enable strict enforcement of the affinity rules you
created.

Procedure
1 In the vSphere Client, display the cluster in the inventory.
2 Right-click the cluster and select Edit Settings.
3 In the left pane of the Cluster Settings dialog box, select VMware DRS and click Advanced Options.
4 In the Option column, enter ForceAffinePoweron.
5 In the Value column, enter 1.
6 Click OK.

Disable Automation Level for DRS
You must set the automation level of all Exchange 2010 mailbox servers to Disabled. When you disable the DRS automation level for a virtual machine, vCenter Server will not migrate the virtual machine to another host or provide migration recommendations for it.
Procedure
1 In the vSphere Client, display the cluster in the inventory.
2 Right-click the cluster and select Edit Settings.
3 In the left pane under VMware DRS, select Virtual Machine Options.
4 Select the Enable individual virtual machine automation levels check box.
5 Change the virtual machine automation level for each Exchange 2010 mailbox server virtual machine in the cluster.
a In the Virtual Machine column, select the virtual machine.
b In the Automation Level column, select Disabled from the drop-down menu.
6 Click OK.

Disable Virtual Machine Restart Priority
Restart priority determines the order in which virtual machines are restarted when the host fails. Because HA does not obey affinity or anti-affinity rules, you must set the VM Restart Priority of  Exchange 2010 vm’s to Disabled.
By default, the same restart priority is used for all virtual machines in a cluster. You can override the default setting for specific virtual machines.

Procedure
1 In the vSphere Client, display the cluster in the inventory.
2 Right-click the cluster and select Edit Settings.
3 In the left pane under VMware HA, click Virtual Machine Options.
4 Change the virtual machine restart priority for each Exchange 2010 virtual machine in the cluster.
a In the Virtual Machine column, select the virtual machine.
b In the VM Restart Priority column, select Disabled from the drop-down menu.
5 Click OK

How to configure a Exchange 2010 virtual machine running on  Hyper-V R2 failover cluster to comply to the Microsoft support policy?

I have not tested this myself. You might put the Exchange 2010 mailbox servers in a separate resource group in Failover cluster Manager. Then select the properties and disable auto Start. This will ensure the virtual machine will not restart after a host failure.

Not sure how to disable Pro Tips for single VM’s.

Read more about this at Jonathan Medd blog:  http://www.jonathanmedd.net/2010/07/exchange-2010-dag-support-within-virtualised-clusters.html

Mind there is a limit to the oversubscription ratio of virtual processors versus logical processors or 2:1 when running Exchange 2010 as a virtual machine.

This means for example your host has 4 quadcore CPU’s (equals 16 logical processors). To have Microsoft support your configuration, you can define a maximum of 32 virtual cpu’s on that host for all virtual machines!

If you decided to install Exchange 2010 on a virtual machine running on Hyper-V, an error might be shown: “An error occurred with error code ‘2147504141’ and message ‘The property cannot be found in the cache.'” To solve this, disable the time synchronizatio between the Hyper-V host and the VM. http://support.microsoft.com/default.aspx?scid=kb;en-us;980050&sd=rss&spid=13965

More info:

In november 2010 VMware published a white paper on virtualization of Exchange Server 201. Read more at this post:
https://up2v.wordpress.com/2010/11/09/microsoft-exchange-server-2010-on-vmware-vsphere-best-practices-guide/

One of the sessions on VMworld 2010 was about virtualizing Exchange Server 2010 mailbox roles. Session EA7849-Design, Deploy, and Optimize Microsoft Exchange Server 2010 was presented at VMworld  2010 USA by Alex Fontana of VMware. See and listen to the presentation by downloading the MPEG4 file from the URL below.
http://www.hotfilesearch.com/download/37990672-EA7849-Design,-Deploy,-and-Optimize-Microsoft-Exchange-Server-2010-on-vSphere.mp4.html

A report on the session can be read here
http://itsjustanotherlayer.com/2010/08/ea7849-exchange-server-2010-on-vsphere/

Info on setting up Windows Guest clustering in vSphere http://www.vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_mscs.pdf

Microsoft Virtualization Best Practises for Exchange 2010.   Interesting presentation of virtualization of Exchange 2010.

Aidan Finn blogpost Exchange Support Policy for Virtualization

Exchange 2010 support in vSphere

Gerben Kloosterman has collected a lot of links to info on Exchange Server 2010 running on vSphere.

VMware blog: Exchange 2010 Disk I/O on vSphere

VMware Blog: Exchange 2010 scale up performance on vSphere

VMware blog : Scaleout performance of Exchange 2010 mailbox server vms on vsphere 4

Measuring the Performance Impact of Exhange 2010 DAG Database Copies   http://communities.vmware.com/blogs/ToddMuirhead/2010/07/26/measuring-the-performance-impact-of-exhange-2010-dag-database-copies

Going Virtual with Exchange 2010  http://www.windowsitpro.com/article/exchange-server/Going-Virtual-with-Exchange-2010/2.aspx

The ClearPath blog contains a lot of useful information on Exchange 2010 running on vSphere.
Part 1 of the  ‘Exchange 2010 on vSphere 4, Best Practices’ series discusses proper Exchange 2010 sizing and requirements around Client Access, Hub Transport, and Mailbox Server roles. In part 2 of Exchange 2010 on vSphere 4, Best Practices, the focus is on  vSphere 4 environment and shows VMware’s and Microsoft’s support Best Practices for ESX cluster and Virtual Machines. Part 3 adds everything up and walks through an example deployment following the guidelines set in the previous installments.

Exchange 2010 on vSphere 4, Best Practices Part 1 by Ryan Williams, Principal Consultant of Clearpath Solutions Group
Exchange 2010 on vSphere 4, Best Practices Part 2 by Ryan Williams, Principal Consultant of Clearpath Solutions Group
Exchange 2010 on vSphere 4, Best Practices Part 3 by Ryan Williams, Principal Consultant of Clearpath Solutions Group

Advertisements

About Marcel van den Berg
I am a technical consultant with a strong focus on server virtualization, desktop virtualization, cloud computing and business continuity/disaster recovery.

2 Responses to Exchange Server 2010 virtualize or not? That’s the question

  1. Pingback: Hyper-V R2 best practices and tips for designing and building your infrastructure « UP2V

  2. Yuri Poloni says:

    Congratulations Marcel. Very good post!!! Much informative…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: