Get VMs by Virtual Port Group with PowerCLI

How to find out what VMs use a particular virtual port group, and quickly?  Well, we have just the thing.  And, it also helps to illustrate the practical use of LinkedViews and the UpdateViewData() method of .NET View objects (about which we posted a while back in Even Faster PowerCLI Code with Get-View, UpdateViewData() and LinkedViews), along with the kind of speed increase that said method brings.

So, the function:

<#  .Description
    Function to get info about what VMs are on a virtual portgroup.  vNugglets, Oct 2013
    Highlights the use of the UpdateViewData() method of a .NET View object
    .Outputs
    PSObject
#>
function Get-VMOnNetworkPortGroup {
    param(
        ## name of network to get; regex pattern
        [parameter(Mandatory=$true)][string]$NetworkName_str
    ) ## end param

    ## get the .NET View objects for the network port groups whose label match the given name
    $arrNetworkViews = Get-View -ViewType Network -Property Name -Filter @{"Name" = $NetworkName_str}
    if (($arrNetworkViews | Measure-Object).Count -eq 0) {Write-Warning "No networks found matching name '$NetworkName_str'"; exit}

    ## get the networks' VMs' names, along with the name of the corresponding VMHost and cluster
    $arrNetworkViews | %{$_.UpdateViewData("Vm.Name","Vm.Runtime.Host.Name","Vm.Runtime.Host.Parent.Name")}
    ## create a new object for each VM on this network
    $arrNetworkViews | %{
        $viewNetwk = $_
        $viewNetwk.LinkedView.Vm | %{
            New-Object -TypeName PSObject -Property @{
                VMName = $_.Name
                NetworkName = $viewNetwk.Name
                VMHost = $_.Runtime.LinkedView.Host.Name
                Cluster = $_.Runtime.LinkedView.Host.LinkedView.Parent.Name
            } | Select-Object VMName,NetworkName,VMHost,Cluster
        } ## end foreach-object
    } ## end foreach-object
} ## end fn

And, some sample usage:

PS vN:\> Get-VMOnNetworkPortGroup -NetworkName "223|237"

VMName        NetworkName   VMHost           Cluster
------        -----------   ------           -------
dubuntu0      223.Prod      esxi01.dom.com   Cluster0
vma5          223.Prod      esxi02.dom.com   Cluster0
vcsa02        223.Prod      esxi02.dom.com   Cluster0
tmpl_test1    237.Dev       esxi12.dom.com   Cluster1
...

This illustrates the use of a regular expression pattern to match virtual networks whose names contain "223" or "237".

And, on the topic of speed increases:

## using standard cmdlets:
PS vN:\> Measure-Command {Get-VirtualPortGroup -Name SomeNetworkName | Get-VM | select name,@{n="VMHostName"; e={$_.VMHost.name}},@{n="ClusterName"; e={$_.VMHost.Parent.Name}}}
...
TotalSeconds      : 25.4870345


## using vNugglets function:
PS vN:\> Measure-Command {Get-VMOnNetworkPortGroup -NetworkName SomeNetworkName}
...
TotalSeconds      : 0.9676149      ## big winner!

So, while similar results can be had with native PowerCLI cmdlets, leveraging this function definitely comes in handy, especially when you are as impatient as we can be here at vNugglets.  Enjoy!

Evacuating VMs and Templates From Datastores

In many environments, ours included, it is not always possible or desired to store all of a VM's virtual disks on the same datastore.  For example, the storage performance may not be ideal if the VM has high I/O requirements, you may have a policy to keep database logs on separate datastores from the actual data files, or you might just have different tiers of storage for your OS disks vs. your data disks.  Whatever the rationale, it is somewhat troublesome if you need to evacuate the datastore for some reason, such as moving to a new storage array.  Why is this?  Consider the following scenario:

Let's assume DATASTORE1 is the datastore that needs to be evacuated in this example.  Initially, you may consider a simple Move-VM command such as this:

Get-Datastore DATASTORE1 | Get-VM | Move-VM -Datastore DATASTORE4

The problem with this command is that it will attempt to move all four VMs to DATASTORE4 but will fail because there isn't enough free space.  At first glance, you may be thinking "What are you talking about?  The VMs on DATASTORE1 only add up to 400GB and there is 500GB free on DATASTORE4!"  Well, the problem with the Move-VM cmdlet is that it will consolidate all the config files and virtual disks that make up a VM at the destination datastore.  If you take a closer look at the diagram, the total capacity needed to host all four VMs is actually 1.4TB.  So Move-VM is not an option here unless we get a larger datastore, but remember, there was a valid reason for splitting the virtual disks across multiple datastores, so we really don't want to lose the VMs' layout on this migration.

To the VMware (VMTN) Communities I went to begin researching this and big surprise, LucD had the answer for someone.  However, this solution had a few issues for us:

1) It does not move any templates that may be located on the datastore you are evacuating.  Surely you use templates, right?

2) It moves the VM's config files even if they are not on the datastore you are evacuating.  Not a huge deal, but some people like to keep the config files on the same datastore as the "OS disk" and this could break that.  For example, if we wanted to evacuate DATASTORE2 instead, VM2 would have not only its 500GB disk moved to the new datastore, but it would also grab the config files from DATASTORE1 as well, which might lead to some confusion since they now reside with the second virtual disk instead of the first (OS disk).

3) If the VM only has its config files stored on the datastore you are evacuating, then we've got a larger problem.  Let's go back to our original example.  We want to migrate the VMs on DATASTORE1 to DATASTORE4, but keep their existing layout so it all fits.  With the script from the Community, it'll work great except on VM3 (and sort of VM4, see above).  With VM3, only the config files are stored there, but because the script sets the "datastore" property of the VirtualMachineRelocateSpec object regardless of where the config files reside, it'll not only move those files, but also all virtual disks associated with that VM because the "disk" property is optional.  Think of it kind of like a "default datastore" to use for migrating the VM and its virtual disks so you don't need to bother specifying the destination datastore on each disk if you desire.  If you do the math on that, it'll move 400GB of virtual disks just for VM3.  That brings the total for all VMs on DATASTORE1 to 800GB, which again, will not fit on DATASTORE4.

So the fix for #3, of course, is to specify the destination datastore for each and every disk--either the new destination datastore if they need moved or the current datastore if they need to stay put.  This is done through the VirtualMachineRelocateSpec object's "disk" property, which is itself an array of VirtualMachineRelocateSpecDiskLocator objects.  Without further adieu, the final code looks like this:

<#
    .Description
    Script to evacuate virtual disks and/or VM config files from a given datastore; does not move the entire VM and all its disks if they reside elsewhere. Created 12-Dec-2012 by vNugglets.com.
    .Example
    EvacuateDatastore.ps1 -SourceDatastore datastoreToEvac -DestDatastore destinationDatastore

    Move virtual disks and/or VM config files (if any) from source datastore to the destination datastore
#>

## Params for source and destination datastore
param(
    ## The name of the source datastore (the one to evacuate).  Required.
    [parameter(Mandatory=$true)][string]$SourceDatastore_str,
    ## The name of the destination datastore.  Required.
    [parameter(Mandatory=$true)][string]$DestDatastore_str
) ## end parameter


## Set proper variable names from the supplied parameters
$strSrcDatastore = $SourceDatastore_str
$strDestDatastore = $DestDatastore_str

## Get the .NET view of the source datastore
$viewSrcDatastore = Get-View -ViewType Datastore -Property Name -Filter @{"Name" = "^${strSrcDatastore}$"}
## Get the linked view that contains the list of VMs on the source datastore
$viewSrcDatastore.UpdateViewData("Vm.Config.Files.VmPathName", "Vm.Config.Hardware.Device", "Vm.Config.Template", "Vm.Runtime.Host", "Vm.Name")
## Get the .NET view of the destination datastore
$viewDestDatastore = Get-View -ViewType Datastore -Property Name -Filter @{"Name" = "^${strDestDatastore}$"}
## Create a VirtualMachineMovePriority object for the RelocateVM task; 0 = defaultPriority, 1 = highPriority, 2 = lowPriority (per http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.VirtualMachine.MovePriority.html)
$specVMMovePriority = New-Object VMware.Vim.VirtualMachineMovePriority -Property @{"value__" = 1}
## Create empty arrays to track templates and VMs
$arrVMList = $arrTemplateList = @()

## For each VM managed object, sort into separate arrays based on whether it is a VM or a template
$viewSrcDatastore.LinkedView.Vm | % {
    ## If object is a template, add to template array
    if ($_.Config.Template -eq "True") {$arrTemplateList += $_}
    ## Else, add it to the VM array
    else {$arrVMList += $_}
}

## For each VM object, initiate the RelocateVM_Task() method; for each template object, initiate the RelocateVM() method
$arrVMList, $arrTemplateList | %{$_} | %{
    $viewVMToMove = $_
    ## Create a VirtualMachineRelocateSpec object for the RelocateVM task
    $specVMRelocate = New-Object Vmware.Vim.VirtualMachineRelocateSpec
    ## Create an array containing all the virtual disks for the current VM/template
    $arrVirtualDisks = $viewVMToMove.Config.Hardware.Device | ?{$_ -is [VMware.Vim.VirtualDisk]}
    ## If the VM/template's config files reside on the source datastore, set this to the destination datastore (if not specified, the config files are not moved)
    if ($viewVMToMove.Config.Files.VmPathName.Split("]")[0].Trim("[") -eq $strSrcDatastore) {
        $specVMRelocate.Datastore = $viewDestDatastore.MoRef
    } ## end if

    ## For each VirtualDisk for this VM/template, make a VirtualMachineRelocateSpecDiskLocator object (to move disks that are on the source datastore, and leave other disks on their current datastore)
    ## But first, make sure the VM/template actually has any disks
    if ($arrVirtualDisks) {
        foreach($oVirtualDisk in $arrVirtualDisks) {
            $oVMReloSpecDiskLocator = New-Object VMware.Vim.VirtualMachineRelocateSpecDiskLocator -Property @{
                ## If this virtual disk's filename matches the source datastore name, set the VMReloSpecDiskLocator Datastore property to the destination datastore's MoRef, else, set this property to the virtual disk's current datastore MoRef
                DataStore = if ($oVirtualDisk.Backing.Filename -match $strSrcDatastore) {$viewDestDatastore.MoRef} else {$oVirtualDisk.Backing.Datastore}
                DiskID = $oVirtualDisk.Key
            } ## end new-object
            $specVMRelocate.disk += $oVMReloSpecDiskLocator
        } ## end foreach
    } ## end if

    ## Determine if template or VM, then perform necessary relocation steps
    if ($viewVMToMove.Config.Template -eq "True") {
        ## Gather necessary objects to mark template as a VM (VMHost where template currently resides and default, root resource pool of the cluster)
        $viewTemplateVMHost = Get-View -Id $_.Runtime.Host -Property Parent
        $viewTemplateResPool = Get-View -ViewType ResourcePool -Property Name -SearchRoot $viewTemplateVMHost.Parent -Filter @{"Name" = "^Resources$"}
        ## Mark the template as a VM
        $_.MarkAsVirtualMachine($viewTemplateResPool.MoRef, $viewTemplateVMHost.MoRef)
        ## Relocate the template synchronously (i.e. one at a time)
        $viewVMToMove.RelocateVM($specVMRelocate, $specVMMovePriority)
        ## Convert VM back to template
        $viewVMToMove.MarkAsTemplate()
    }
    else {
        ## Initiate the RelocateVM task (asynchronously)
        $viewVMToMove.RelocateVM_Task($specVMRelocate, $specVMMovePriority)
    }
} ## end foreach-object

To use the script, you'll need to provide the source and destination datastore as parameters as such:

PS vNuggs:\> .\EvacuateDatastore.ps1 -SourceDatastore DATASTORE1 -DestDatastore DATASTORE4

Many of the comments in the code are self-explanatory, but here are some additional details around the more important/complex parts:

Lines 35-40: In this section, we determine whether the objects from the datastore's linked view are standard VM managed objects or whether they are actually a template and put them into separate arrays.

Line 43: This is where we pipe the two arrays into a ForEach-Object loop to start the process of gathering the necessary data to relocate the VM/template to the new datastore.  We chose to begin with the VMs first because we eventually want to kick them off as vCenter tasks so that we can then focus on the templates.

Line 50: This is the part that combats problem #2 above.  Here, we parse the datastore name out of the fully-qualified path to the VM/template's config file and if it matches the source datastore, then that's the only time we set the "datastore" property of the VirtualMachineRelocateSpec object to the destination datastore as that's the only time we want to move the config files.

Lines 56-65: Now the VirtualMachineRelocateSpecDiskLocator object is populated accordingly--if the disk is on the source datastore, then its "datastore" property is set to the destination datastore so it gets moved, otherwise it is set to its current datastore so it does not get moved.  Line 63 adds it to the "disk" property of the VirtualMachineRelocateSpec object and everything repeats for the next disk if more exist.

Lines 68-78: On line 68 we have to check if the current VM object is really a VM or a template, even though we already did this earlier.  This is better than having to duplicate all the code from lines 43-65, however.  Then lines 70-73 are how we deal with problem #1 from above.  Since we determined this object was a template, we must convert it to a VM in order to relocate it to a different datastore.  It is a shame VMware still doesn't allow us to move templates for some reason.  Finally, line 75 calls the RelocateVM() method of the VM managed object and begins the relocation.  It is important to note that this occurs in a synchronous manner (i.e. one template at a time).  As you'll see in the next section, we don't do this if it is a VM object, but we really have no choice when dealing with templates because we need to wait for the relocation to complete before we can convert the VM back to a template.  This occurs on line 77.

Lines 79-82: In this section, namely line 81, we call the RelocateVM_Task() method this time since this object is a VM, not a template.  This way we kick off "RelocateVM" tasks asynchronously and let vCenter decide how many it can handle at once, which as of vSphere 5.0 and 5.1 is eight per datastore and two per host.  In addition, we piped the VM objects into the loop first so that we could kick them all off and then focus on the templates one by one.  But, if you are concerned that your storage array won't be able to handle the load of multiple storage migrations, simply change line 81 to look like line 75 and it'll behave synchronously like the templates do.

If you've read this far, you may be wondering what use this sort of script has in the world of SDRS (Storage DRS).  Yes, SDRS has features like datastore maintenance mode that would prevent the need for some of this type of work, but not everyone has SDRS available to them; either because they haven't found the time to implement it yet, haven't upgraded to vSphere 5.x yet, or just don't own the Enterprise Plus edition, so this script is likely to be of use to many people still. We hope so.

Even Faster PowerCLI Code with Get-View, UpdateViewData() and LinkedViews

We are always looking for speed, speed, speed in our PowerShell / PowerCLI scripts.  When properly used, the Get-View cmdlet is known to be far faster than using other PowerCLI cmdlets for getting info (such as Get-VM, Get-VMHost, Get-Datastore, etc.).  It can take quite a bit longer to write a script using Get-View versus using other cmdlets, but the speed payoff is usually well worth the additional upfront time investement.  VMware has provided a way to speed things up even more.

A while back (Aug 2011), the official PowerCLI blog had a post about how to "Optimize the performance of PowerCLI's views".  The technique is centered around using the UpdateViewData() method of a "view" of a managed object (".NET View object") to retrieve nested views of managed objects.

The UpdateViewData() method had already been able to selectively update properties of the given managed object view, but one would have to use Get-View again to retrieve views of nested managed objects that are properties of that parent managed object.  "So, what?"  So, speed!

Example 0:  Get guest IP info for VMs using given VM network

Say you wanted to get the IPs configured on VMs that are connected to particular VM networks.  This example goes about doing so by getting said network(s), getting the VMs connected to them, and then getting guest IP info.

You could use the following standard way using multiple Get-View calls:

## the network name pattern to use for getting vitual networks
$strNetworkNamePattern = "MyVMNetwork1711"
## get the virtual networks whose name match this pattern
$arrNetworkViews = Get-View -ViewType Network -Property Name,VM -Filter @{"Name" = $strNetworkNamePattern}

## get the VM managed objects with a couple of properties
$arrVMsUsingThis = $arrNetworkViews | ?{$_.Vm -ne $null} | %{Get-View -Property Name,Guest.Net $_.Vm}

## get info about IPs used on this/these network(s)
$arrVMsUsingThis | %{
   $strVMName = $_.Name
   ## for each Guest.Net where the network name is like the given network name
   $_.Guest.Net | ?{$_.Network -like "*$strNetworkNamePattern*"} | %{
       ## get IPs
       $_.IpAddress | %{
           New-Object -TypeName PSObject -Property @{
               VMName = $strVMName
               IPAddr = $_
           } ## end new-object
       } ## end %
   } ## end %
} ## end %

Or, using the UpdateViewData() method, the code would be:

## the network name pattern to use for getting vitual networks
$strNetworkNamePattern = "MyVMNetwork1711"
## get the virtual networks whose name match this pattern
$arrNetworkViews = Get-View -ViewType Network -Property Name -Filter @{"Name" = $strNetworkNamePattern}

## get a couple of properties for the VMs using this network
$arrNetworkViews | %{$_.UpdateViewData("VM.Name","VM.Guest.Net")}

## get info about IPs used on this/these network(s)
$arrNetworkViews | %{$_.LinkedView.Vm} | %{
   $strVMName = $_.Name
   ## for each Guest.Net where the network name is like the given network name
   $_.Guest.Net | ?{$_.Network -like "*$strNetworkNamePattern*"} | %{
       ## get IPs
       $_.IpAddress | %{
           New-Object -TypeName PSObject -Property @{
               VMName = $strVMName
               IPAddr = $_
           } ## end new-object
       } ## end %
   } ## end %
} ## end %

The only slight differences in the code in the two ways:

• Line 04 in each:  the first way gets an additional property, "VM", for later use
• Line 07 in each:  the first way uses and additional Get-View call, whereas the second way uses the UpdateViewData() method
• Line 10 in each:  both access an array of VMs connected to the given VM network, but getting to those VM arrays is a touch different (the "LinkedView" property in the second way is due to having used the UpdateViewData() method)

This runs reasonably quickly both ways when just getting information for VMs on one VM network.  But, to really illustrate the potential speed increases, I tested the main difference (line 07) with not just one or two networks, but with about 30 networks and with about 160 networks.  The results are impressive.  Again, this was measuring just the central point of each way:  the amount of time for the code on line 07 in each way above to get the VM info from vCenter (first via an additional Get-View, and then via UpdateViewData()).

Get-View vs. UpdateViewData() -- Speed testing results in a couple of different environment sizes:

Method used	Num. networks	Num. netwk connections	Time to run (sec)	Total Get-View calls
additional Get-View call	33	432	95.3	34
UpdateViewData()	33	432	7.75	1
additional Get-View call	159	3705	811.4	160
UpdateViewData()	159	3705	39.61	1

Mhmm.  7.75 seconds vs. 95.3 seconds in a small-ish environment -- more than twelve (12) times faster for UpdateViewData().  And 40 seconds vs. 811 seconds in a larger environment -- more than twenty (20) times faster!

Example 1:  Get VMHosts' FirewallSystems to refresh firewall info

Another example.  AC recently had needed to refresh the VMHost firewall info on hosts.  This stemmed from a bug that AC ran into in which the host firewall info is gathered during the host boot process before the firewall services are fully initiated.

Next is the overall code for each way, and then some speed comparisons of each method.  Note, we originally wrote this to only refresh firewall info for VMHosts in one cluster at a time, but to have a bit larger set of operations so as to better illustrate the speed differences, the following code performs the actions for all clusters, one cluster at a time.

Traditional way, using multiple Get-View calls:

## get the Cluster managed objects
Get-View -ViewType ClusterComputeResource -Property Name,Host | %{
   ## get all of this cluster's HostSystem managed objects
   $arrHostViews = Get-View $_.Host -Property ConfigManager.FirewallSystem
   ## for each host in the given cluster, get the FirewallSystem managed object
   foreach ($viewHost in $arrHostViews) {
       $viewFirewallSystem = Get-View $viewHost.ConfigManager.FirewallSystem -Property FirewallInfo.DefaultPolicy
       ## call RefreshFirewall() method for this host's FirewallSystem
       $viewFirewallSystem.RefreshFirewall()
   } ## end %
} ## end %

And, the UpdateViewData() way:

## get the Cluster managed objects
$arrClusterViews = Get-View -ViewType ClusterComputeResource -Property Name
## update the .NET View data with a select FirewallSystem sub-property
$arrClusterViews | %{$_.UpdateViewData("Host.ConfigManager.FirewallSystem.FirewallInfo.DefaultPolicy")}
## for each host in the given cluster, call RefreshFirewall() method
$arrClusterViews | %{$_.LinkedView.Host} | % {$_.ConfigManager.LinkedView.FirewallSystem.RefreshFirewall()}

The differences in code between the two ways:

• Lines 04-07 in 1st way vs. Line 04 in 2nd way:  The first way uses the standard way of additionally calling Get-View for each cluster to get the HostSystems, and then again for each HostSystem to get the FirewallSystem objects.  The second way uses one UpdateViewData() call to make the FirewallSystem managed objects available from the original array of cluster managed objects.

To see the speed differences for the main info-gathering portion of each way, I measured these two pieces mentioned above:  lines 04-07 for the first way, and line 04 for the second way.  UpdateViewData() wins.

Get-View vs. UpdateViewData() -- Speed testing results in a couple of different environment sizes:

Method used	Num. clusters	Num. HostSystems	Time to run (sec)	Total Get-View calls
additional Get-View	5	21	9.47	27
UpdateViewData()	5	21	1.39	1
additional Get-View	36	191	70.85	228
UpdateViewData()	36	191	8.20	1

The speed gain comes, in part, from the sheer number of calls being made that require a round trip request to the given vCenter server(s).  To get additional managed object info using Get-View, there is a Get-View call involved for every [group of] managed object references (MORefs).  Conversely, there is but one (1) call to communicate with vCenter for all "child" managed objects involved when UpdateViewData() is used.

This is not the most obvious route at first, but eventually you can see how to use the method to update managed object view data, and then access the nested managed objects' views using "LinkedView" objects that then reside "in" the parent managed object as properties.

The reward for successfully employing the technique:  speed.  The PowerCLI blog post reports a 100x speed increase in the given example versus using multiple Get-View calls to get nested managed object views.  100x -- that is speeding up a script that takes five (5) minutes to take only three (3) seconds -- FaF!  This speed increase varies based on how many Get-View calls are saved/avoided, what they are retrieving, etc, but that kind of potential speed gain demands attention.

For some further info:  I used this technique in a VMware communities thread about VM portgroup inventory (yes -- December 2011; it has taken me a while to get this post together).  It shows other examples of using the resulting LinkedView properties.  And, there is good info in the PowerCLI lab guide from the 2011 VMworld labs.  The PowerCLI Blog post about PowerCLI Lab at VMWorld 2011 links to the lab manual PDF.  See pages 91-93.

How much faster is your script about to be?