vSphere 5 versus Windows Server 2012 Hyper-V: live migrations

This is the first in a series of postings I will do on VMware vSphere 5 and Windows Server 2012 Hyper-V. Goal of the postings is to give a non-biased overview on features of two main players in the server virtualization market: VMware vSphere and Microsoft Hyper-V.

Other blogs in the serie are:
vSphere 5 versus Windows Server 2012 Hyper-V: storage integration
vSphere 5 versus Windows Server 2012 Hyper-V:management
vSphere 5 versus Windows Server 2012 Hyper-V: high available VMs
vSphere 5 versus Windows Server 2012 Hyper-V: Resource metering for chargeback
vSphere 5 versus Windows Server 2012 Hyper-V: costs
vSphere 5 versus Windows Server 2012 Hyper-V: hybride cloud

Lets start this series with an overview of the ability to move virtual machines from one host to another. VMware names this feature vMotion, Microsoft names it Live Migration.
Microsoft claims that Windows Server 2012 Hyper-V offers an unlimted number of concurrent Live Migrations. VMware vSphere can handle a maximum 8 concurrent vMotions on a 10 Gbps network.

vMotion/Live migration can be used for several use cases:
1. planned maintenance of a host. The host needs to be shutdown to add extra hardware or to install service packs or other software updates on the hypervisor which requires a reboot.
2. load balancing: when a host cannot deliver the resources a virtual machine needs. One or more virtual machines will be moved to another host.
3. critical hardware error: when a host is likely to fail in the very near future due to for example failure of a coolingfan and the admin wants to evacuate the virtual machines on that host as soon as possible.

Having the possibility to have an unlimited number of concurrent Live Migrations is only beneficial in use case number 3: a host is about the fail and you want your VMs to be evacutated as soon as possible. In the other cases it does not matter much if migration takes an hour or so longer.

Wait a moment:  does Unlimited also mean that virtual machines are moved faster than when a limited number of migrations is used?

NO! There is a reason why there is a limit even on 10 Gbps network connections. Live migration/vMotion mean the content of the internal memory of the to be moved VMs needs to be copied over to another host and kept in sync before the actual move. This can consume quite a bit of bandwidth depending on the internal memory usage of the VM. If an unlimited number of VMs are concurrently migrated to another host to might lead to congestions and much longer migration time, timeouts or even failures.

See the opinions of other bloggers like Chris Wahl in his posting titled From The Bad Ideas Department: Unlimited VM Migrations

Aidan Finn , which blogs exclusively about Microsoft Hyper-V did some benchmarking to find the sweet spot of the number of concurrent Live Migrations of Windows Server 2012 Hyper-V. In his blog he writes he found out that 20 concurrent Live Migrations took the least amount of time to complete using VMs with 512 MB internal memory.

So more or unlimited number of concurrent Live Migrations does not mean it is faster than a limit on the number of concurrent migrations.

Eric Gray, a VMware employee with a personal blog, responds to the Unlimited Live migration claim in his blog titled Hyper-V 3 Offers Unlimited Live Migrations — Kind of

I believe the Unlimited number of Live Migrations is a marketing term. In real life using real workloads under the same conditions the number of concurrent Live Migration delivering the shortest migration time for Windows Server 2012 Hyper-V will most likely be the same as for vSphere’s vMotion feature.


About Marcel van den Berg
I am a technical consultant with a strong focus on server virtualization, desktop virtualization, cloud computing and business continuity/disaster recovery.

2 Responses to vSphere 5 versus Windows Server 2012 Hyper-V: live migrations

  1. Aidan Finn says:

    Having being quoted, I thought I’d respond here.

    I think you have under sold the desire to do as many live migrations as possible. In scenario (1) you definitely want to get virtual machines off a host as quickly as possible. If I’m planning on doing some work to a host with p to 4 TB of RAM in it, I don’t want to be waiting 3 days.

    In the test that you quoted, 20 simultaneous live migrations (over twice that possible on vSphere with the same networking) was the maximum for my configuration in that test. Once again, if I’m running hosts with 1 TB RAM or even 4 TB RAM, then I want to be able to vacate those hosts as quickly as possible. In that enterprise configuration, even 10 GbE won’t be enough, and I’ll use InfiniBand. I’m thinking that with 56 Gbps I’m going to increase my simultaneous live migrations way beyond 20 … and even further beyond the best that the competition can offer.

    • Aidan, what’s bugging people is that Microsoft likes to pretend they have an advantage because obviously, “unlimited” is a lot more than just 4 or 8.
      But what matters are not how many cars you can put on each respective highway at one time, but how fast they (or the last one) can reach their destination.
      You demonstrated that ~20 was the sweet spot in your very specific test case using Hyper-V 2012, and that’s cool. However, this holds no implications on how vSphere with its maximum of 8 simultaneous migrations would fare in the same test case whatsoever.

      Also, if you reach a sweet spot of 20 in an environment with many small idle VMs and a huge Migration bandwidth of 40Gb, I can’t help but assume that in a real environment, even with Infiniband, you will never get “way beyond 20 … and even further beyond the best that the competition can offer”. I don’t see how your 40Gbps network could have been anywhere near saturated as you pushed just 30GiB of idle RAM through it in over 70 seconds, making about 3.5Gbps on average. I’d love to see some data on the actual network utilization.

      And because some people seem to imply, or have the impression that on vSphere, you would have to migrate VMs in batchs of 4/8 separately or something, I also just want to mention that vSphere simply queues the jobs up and it’s about simultaneous executions.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: