By default, when instances are created, they are spread in the cluster. So the first VM will be in  the first compute node, the second one in the second compute node and so on.  This will ensure that all nodes are properly used up and filled up.

Suppose, C11 has 8 CPUs and C12 has 8 CPUs each.

Suppose you have the following flavors:

  • my-4cpu-flavor
  • my-6cpu-flavor

In default OpenStack setup (spreading)

Task Node CPU
Count
CPU
Requested
CPU
Total
CPU
Remaining
Add compute c11 8 8 8
Add compute c12 8 16 16
add new instance:  my-4cpu-flavor c11 4 12
add new instance: my-4cpu-flavor c12 4 8
add new instance: my-6cpu-flavor host not found

As you see, the dashboard will show that you have 8 CPU remaining, but due to spreading, you cannot spawn a new VM.
I have seen similar problem even when you have 50+ nodes in the cluster, shows 200 vcpu free, but you are unable to launch an instance because they are evenly spread, so combined it has 200 vcpu free, but no node has enough cpu to handle 6 vcpus.

This also makes your capacity planning tricky, because do not know how much you are growing.

To change this behaviour, you need to edit your nova.conf and enable this setting:

What this does is now the nova scheduler will start filling up on the first available node as much as possible before moving on to the next node. So in our above example, the first 2 4cpu flavor will end up in c11 and the next one 6cpu will be on c12.

For customers, you can point them to this document where they can setup anti-affinity rules to ensure that their instances do not land on the same node.

About The Author