Cluster Admission Control in vSphere 6.5/6.5U1

vSphere HA uses admission control to ensure that sufficient resources are reserved for virtual machine recovery when a host fails. The basis for vSphere HA admission control is how many host failures your cluster is allowed to tolerate and still guarantee failover for the VMs on to the remaining hosts. The default Admission Control policy has changed from Slot Policy (Default until 6.5), to ‘Cluster Resource Percentage’. VMware had found that very few people were actually using slot policy, and those that were, were not using it correctly and it also involved some manual calculations when hosts were added or removed.

Admission control imposes constraints on resource usage. Any action that might violate these constraints is not permitted. Actions that might be disallowed include the following examples:

• Powering on a virtual machine
• Migrating a virtual machine
• Increasing the CPU or memory reservation of a virtual machine

Computing the Current Failover Capacity

The total resource requirements for the powered-on virtual machines is comprised of two components, CPU and memory. vSphere HA calculates these values.

• The CPU component by summing the CPU reservations of the powered-on virtual machines. If you have not specified a CPU reservation for a virtual machine, it is assigned a default value of 32MHz (this value can be changed using the das.vmcpuminmhz advanced option.)
• The memory component by summing the memory reservation (plus memory overhead) of each powered-on virtual machine.

The total host resources available for virtual machines is calculated by adding the hosts’ CPU and memory resources. These amounts are those contained in the host’s root resource pool, not the total physical resources of the host. Resources being used for virtualization purposes are not included. Only hosts that are connected, not in maintenance mode, and have no vSphere HA errors are considered.

The Current CPU Failover Capacity is computed by subtracting the total CPU resource requirements from the total host CPU resources and dividing the result by the total host CPU resources.

The Current Memory Failover Capacity is computed by subtracting the total Memory resource requirements from the total host memory resources and dividing the result by the total host Memory resources

Host Failure Cluster Tolerates 

This option allows you to define the number of ESXi hosts tolerate for failures. vSphere HA will automatically calculate a percentage of resources to set by applying the “Percentage of Cluster Resources” (Default option in vSphere 6.5) admission control policy. Resources required for failover capacity is now directly related to the Host failures cluster tolerates option. In the example below, there are 2 ESXi hosts in the cluster and I have configured “Host failures cluster tolerates” value as “1”. HA will then automatically reserve 50% of Memory and 50% of CPU for the failover capacity. If you have a 4-host cluster and FTT=1 then it will calculate a 25% reservation. HA Slot policy used to be the default admission control policy. With vSphere 6.5, the default admission control policy is now “Cluster resource Percentage”.

If you have add or remove ESXi hosts in the cluster, the percentage of failover capacity will be automatically recalculated.

You have the option to Override the failover capacity by the number of host failures cluster tolerates by selecting the Override option and specify % for CPU and Memory.

 

Define host failover capacity by HA Slot Policy

You can also have option to choose “Slot Policy”.  This is the default option prior to vSphere 6.5. Slot Size is defined as the memory and CPU resources that satisfy the reservation requirements for any powered-on virtual machines in the HA cluster. You have 2 options under Slot Policy:

• Cover All powered-on Virtual Machines

It calculates the slot size based on the maximum CPU/Memory reservation and overhead of all powered-on virtual machines in the cluster but will be skewed by reservations on VMs which is why we have the setting below to override calculations being based on large reservations.

• Fixed Slot Size

You can explicitly specify the fixed slot size

 

Define host failover capacity by Dedicated Failover Hosts

This option allows you to define a dedicated ESXi host in the cluster as failover hosts for the HA cluster.  That dedicated failover host will not run virtual machines unless vSphere HA needs to recover from a failed host. However this is a waste of a whole host and not generally used unless you have the ability and capacity in your datacenter to keep a spare host aside just in case.

 

VM resource reduction event threshold – Performance Degradation Tolerance

The reserved capacity used by admission control ensures that all configured reservations will continue to be honored after a host failure.  If the reservations are not used in some environments, the performance of the cluster could be impacted.  This new setting called “VM resource reduction event threshold” defines how much of a performance impact is tolerated and will issue a warning if the consumed resources are higher than the reserved resources.

0% – Raises a warning if there is insufficient failover capacity to guarantee the same performance after VM’s restart.

100% -Warning is disabled.

 

Example

400GB of memory available in a 4 node cluster
1 host failure to tolerate specifed
310GB of memory actively used by VMs
0% resource reduction tolerated

This results in the following:
400GB – 100GB (1 host worth of memory) = 300GB
We have 310GB of memory actively used, with 0% resource reduction to tolerate
310GB needed, 300GB available after failure > A warning will be issued.

Summary

Just some general points I’ve seen lately. I’ll update them if anything else comes up which is interesting.

• In terms of larger VMs skewing the options, any CPU or Memory VM reservation will be taken into account including the defaults taken for any VMs which do not have reservations.
• In regards to reservations, they will only come into play when the system is under contention anyway. VMware is very good at managing resources. However if you do want to use them, I would advise monitoring the peak usage of CPU and RAM of the VMs you want a reservation on first so you can assign an accurate reservation.
• Slot Policy is no longer the default option and is easy to set incorrectly so be careful with this option. You can create extra work for yourself but it is useful.
• Use the Host failures to tolerate policy (the recommended policy now)
• It may be worth setting the VM resource reduction event threshold so you are warned of any potential performance problems. Setting it to 0% will generate a warning if Admission Control thinks there is insufficient failover capacity to ensure the same performance after VMs are failed over. This is achieved through monitoring of actual usage of CPU and RAM resources. In conjunction with the Host Failures to Tolerate policy which just calculates using reservations, default cpu/ram values and overhead for its calculations, these 2 settings combined give a very useful monitoring aspect for your cluster.