This post will show you how to configure datastores allowed for vSphere Cluster Service VM’s using a PowerCLI script. As a side effect you will also learn a little bit about finding your way around the vSphere Web Services API.
If you are already familiar with this you can of course jump straight to the finished script.
Introduction
In vSphere 7.0 Update 3 vSphere Cluster Services (vCLS) have been introduced.
VMWare Docs describe the construct as follows:
vCLS uses agent virtual machines to maintain cluster services health. The vCLS agent virtual machines (vCLS VMs) are created when you add hosts to clusters. Up to three vCLS VMs are required to run in each vSphere cluster, distributed within a cluster. vCLS is also activated on clusters which contain only one or two hosts. In these clusters the number of vCLS VMs is one and two, respectively.
In short, these automaticly created VM’s are based on a very stripped down PhotonOS. They get managed by vSphere itself and are only visible if you are logged in as administrator@vsphere.local. You will be warned if you try and interact with the vCLS VM’s manually as this could potentially mess up functionality of your cluster.
Situation
Once you build a fresh vSphere 7 (or higher) environment with a cluster object in it you will immediatly see the vSphere Cluster Services Virtual Machines getting created. They will live in a folder called vCLS and start with the prefix “vCLS”. If you inspect the datastores they live on, you will see that vSphere selected those by choosing one of the available shared datastores.
The problem here might be: It will for example also happily choose NFS datastores if available. In my case this is what happened. As these are also available via SMB this is probably not a good idea, as people might mess with the VM folder if they don’t know its origin.
Goal
We want vSphere Cluster Service to know on which datastores it can (or can not) create those VM’s
This could be configured via UI in this place:
But what if you have recently done an upgrade to vSphere 7 or higher and you have to deal with a lot of datastores and clusters ? That’s were a little PowerCLI script could come in handy, and this is what we’re going to do here.
Solution using PowerCLI
In my case I wanted to configure the vCLS VM placement rule so they could live on any shared datastore, except NFS. We are going to break down all the pieces to end up with the finished script.
Reading the docs
First we dive into the vSphere Web Services API to look up the object we are interested in. As we are dealing with a cluster object, the configuration for the Cluster Services VM’s might be in there.
Looking at the “ClusterConfigSpecEx” data object there is a property that looks promising: “systemVMsConfig”. This is described as “Configuration for vCLS system VMs deployment"and it’s type is “ClusterSystemVMsConfigSpec”.
Now “ClusterSystemVMsConfigSpec” holds the property “allowedDatastores” of type “ClusterDatastoreUpdateSpec” which seems to be exactly what we are looking for:
Note that the property “allowedDatastores” is actually an array of “ClusterDatastoreUpdateSpec”. This becomes important once we build this property.
So it looks like we have to piece together a “ClusterConfigSpecEx” to be able to apply this to our cluster then…
Building the script
Outline
A rough outline of what we need to build could look something like this:
- Initialize
- Get target datastores
- Loop over datastores
- Build ClusterConfigSpecEx Object in loop
- Apply ClusterConfigSpecEx to our cluster
Initialize
There are some objects that have to be initialized so we can work with them later on:
$clspec = New-Object VMware.Vim.ClusterConfigSpecEx
$clspec.DrsConfig = New-Object VMware.Vim.ClusterDrsConfigInfo
$clspec.SystemVMsConfig = New-Object VMware.Vim.ClusterSystemVMsConfigSpec
(About the DrsConfig: I know we are not going to use that, but it has to be in there even if it’s empty.)
Get target datastores
We need to get all the datastores to have them available once we are going to reconfigure the cluster. As mentioned above, what we want is: All shared datastores except NFS
$datastores = get-cluster -name $clusterName | get-datastore | Where-Object {$_.ExtensionData.Summary.MultipleHostAccess -eq 'true' -and $_.type -ne "NFS"}
Loop over datastores / Build ClusterConfigSpecEx
Once we have our datastore objects in an array we are going to loop over it by building a ClusterDatastoreUpdateSpec on each iteration and pushing it to the SystemVMsConfig.AllowedDatastores array (Rember our note from above ?).
$datastores | ForEach-Object{
$dsspec = New-Object VMware.Vim.ClusterDatastoreUpdateSpec
$dsspec.Datastore = New-Object VMware.Vim.ManagedObjectReference
$dsspec.Datastore.Type = "Datastore"
$dsspec.Operation = "add"
$dsspec.Datastore.Value = $_.Extensiondata.MoRef.Value
$clspec.SystemVMsConfig.AllowedDatastores += $dsspec
}
Apply ClusterConfigSpecEx to our cluster
Now that our ClusterConfigSpecEx is in $clspec it’s time to apply this to our cluster by reconfiguring it:
(Get-Cluster -name $clusterName).ExtensionData.ReconfigureComputeResource($clspec,$true)
Quick note on this one: If you are not too familiar with things like this, always try to understand what the reconfigure task will do. In this case passing either $true or $false as the second parameter can influence the rest of your day drasticly …
Finished Script
#################################################
### Connect vCenter
#################################################
$vCenterServer = Read-Host -Prompt 'Enter the FQDN of the vCenter Server you want to connect to (ex. vcenter.domain.com)'
$credentials=Get-Credential -Message "Enter your Credentials"
Try {
Connect-VIServer -Server $vCenterServer -Credential $credentials -ErrorAction Stop | Out-Null
}
Catch {
Write-Host -ForegroundColor Red -BackgroundColor Black "Could not connect to the vCenter Server [$vCenterServer]." `n
Exit
}
#################################################
### Init Variables
#################################################
$clusterName = "mycluster"
$clspec = New-Object VMware.Vim.ClusterConfigSpecEx
$clspec.DrsConfig = New-Object VMware.Vim.ClusterDrsConfigInfo
$clspec.SystemVMsConfig = New-Object VMware.Vim.ClusterSystemVMsConfigSpec
#################################################
### Code
#################################################
## Get all shared datastores attached to cluster except local and NFS
$datastores = get-cluster -name $clusterName | get-datastore | Where-Object {$_.ExtensionData.Summary.MultipleHostAccess -eq 'true' -and $_.type -ne "NFS"}
Write-Host ("Datastores to be configured for VCLS VM placement: " + $datastores)
## Iterate datastore array, build cluster spec object for reconfig task
$datastores | ForEach-Object{
$dsspec = New-Object VMware.Vim.ClusterDatastoreUpdateSpec
$dsspec.Datastore = New-Object VMware.Vim.ManagedObjectReference
$dsspec.Datastore.Type = "Datastore"
$dsspec.Operation = "add"
$dsspec.Datastore.Value = $_.Extensiondata.MoRef.Value
$clspec.SystemVMsConfig.AllowedDatastores += $dsspec
}
## Reconfig cluster
(Get-Cluster -name $clusterName).ExtensionData.ReconfigureComputeResource($clspec,$true)
#################################################
### Disconnect vCenter
#################################################
Disconnect-VIServer -server $vCenterServer -confirm:$false
Conclusion
This article showed you one way of configuring vCLS VM placement. Depending on your use case you might approach a few things in a different way. You could for example:
- Get all of your clusters and loop over them to have everything configured in one go.
- Go for not allowing datastores rather than explicitly allowing them. In this case you will use “notAllowedDatastores” in ClusterSystemVMsConfigSpec and invert the datastore selection.
I hope you enjoyed this post. Feel free to share this with the social icons below if you think this could be interesting for your contacts.