How to enable nested virtualization in Azure

We have already mentioned new Azure VM series Dv3 and Ev3 which enable running VMs inside Azure VMs or just nested virtualization. Today we are going to get it configured and to run our first nested VM in Azure.

But before we start, let’s review some Dv3 and Ev3 facts:

  • they introduce Hyper-Threading Technology running on the Intel® Broadwell E5-2673 v4 2.3GHz processor and Intel® Haswell 2.4 GHz E5-2673 v3
  • they made shift from physical core to virtual CPUs (thanks to HT technology) to support larger VM sizes
  • they are the first Azure VMs running on Windows Server 2016 hosts
  • Dv3 VMs are up to 64 vCPUs and 256 GB RAM
  • Ev3 VMs are up to 64 vCPUs and 432 Gb RAM
  • they are currently available only for certain regions (West Europe, US East, US West 2, Asia Pacific Southeast)
  • they already come with ExposeVirtualizationExtensions enabled. we don’t need to enable CPU extensions as we have to do for on-premises WS2016 hosts

To get started with “nesting” you need to create one or more Dv3/Ev3 VMs in Azure within compatible region. For quick demo purposes, I created D2S_V3  VM with Windows Server 2016 DC , standard managed disk with no data disks attached.

TIP: actually you can , for instance, create 2 or more VMs , add data disks and configure storage spaces between them to achieve higher IO performance.

Then you need to install Hyper-V role and restart VM to apply changes

Install-WindowsFeature Hyper-V -IncludeManagementTools -Restart

nested virtualization azure 1

Verify that Hyper-V role is installed and add internal switch. New adapter “vEthernet (switchname)” will be created under network connections list (ncpa.cpl)

Define a new IP address for this adapter (I’m using 192.168.0.0/24 subnet).  This network will be used as a NAT gateway for new VMs in order to allow internet access from nested VMs.

#Check Hyper-V role state
Get-WindowsFeature Hyper-V|ft InstallState, PostConfigurationNeeded

#Add new internal switch
New-VMSwitch -SwitchName "NSW01" -SwitchType Internal

# IP Configuration for vNIC
New-NetIPAddress -InterfaceAlias "vEthernet (NSW01)" -IPAddress 192.168.0.23 -PrefixLength 24

nested virtualization azure 2

Configure NAT rule to provide “access” to our nested VMs

New-NetNat -Name Nat_VM -InternalIPInterfaceAddressPrefix 192.168.0.0/24

image

Now our nested VMs can assign IP addresses from 192.168.0.0/24 subnet  (manual assignment). If you want to have dynamic IP assignment – create add. VM and configure DHCP. Continue reading “How to enable nested virtualization in Azure”

FAQ: Windows Server 2016 and Hyper-V Integration Services

I’m frequently receiving questions about what’s wrong with Windows Server 2016 and how to manage Hyper-V IS. I’d like to discuss it in a FAQ manner. If your question is not answered, just feel free to leave a comment

Where is the integration services setup disk (VmGuest.iso)?

All of us remember times when we had to insert ISO which was available on Hyper-V host , run it on each VMs and check it’s version from time to time for keeping them up-to-date (yes, vmguest.iso was updated via Windows Update).

The process of Hyper-V IS upgrading has been changed since Windows Server 2016. Now VMs get IS updates directly from Windows Update (WU). ISO is not included and it’s not needed anymore.

It’s completely true if you have new OS inside VMs (Windows 2012 R2/Windows 8.1 and later). Hyper-V IS automatically get updates from WU in these OSes. No actions required.

We have another picture for older OSes (Windows Server 2012/Windows 8 and older). For such OSes, you need to verify that Data Exchange Service (DES) is turned on and running.

DES provides shared registry values (HostSystemOSMajor and HostSystemOSMinor under the hive HKLM\SOFTWARE\Microsoft\Virtual Machine\Guest\Parameters) between host and VMs for determining IS applicability. These values should be existed in VM in order to get IS updates from WU. If DES has never been enabled on VMs – IS won’t be provided.

hypervregs

Hack: you can edit/add these parameters and set values manually (of course, it’s unsupported)

Example of outdated IS in VM with WS2012R2 on WS2016 host (DES provided values from VM, compared them with host’s and they are not the same):

image

Linux Guests? Built-In or  Linux Integration Services are required

Unsupported Windows Server 2003? Integration services setup disk is required. Can be copied from WS2012R2 (download link is below). Better option: plan migration to newer OS

How to install IS manually?

Use this KB if DES can’t be enabled or WU is unavailable:

Hyper-V integration components update for Windows virtual machines that are running on a Windows 10 or Windows Server 2016-based host (IS  ver. 6.3.9600.17903) + IS update for Windows Server Guest OSes (IS ver. 6.3.9600.18080)

Update: IS was updated by the June’s month rollup to version 6.3.9600.18692

Note: To install this update, you must have April 2014, update rollup for Windows RT 8.1, Windows 8.1, and Windows Server 2012 R2 (2919355) installed in Windows 8.1. Or, install Service Pack 1 for Windows 7.

To install IS (CAB-file) use PowerShell:

Add-WindowsPackage -PackagePath pathtocab -Online

If you want to do offline VM servicing : read guide from Sarah Cooley

TIP: IS with later version is available from uploaded VMGuest.iso below (use it to simplify overall process)

Where can I find ISO if I haven’t got Windows Server 2012 R2 installed?

I’ve uploaded Hyper-V Integrations Services ISO for you. It was taken from Windows Server 2012 R2 (up-to-date, 02/20/17 . IS version: 6.3.9600.18398). You can use this ISO to install Integration Services in “prehistoric” guest OSes (but again..migration to newer OS is better option for such VMs) on Windows Server 2016 or with hosts running on Windows 2012 R2/Windows 8.0/8.1 Download link for VMGuest.ISO

I’m querying Integration Services version from PowerShell but it shows me 0 for all VMs. What’s wrong?

I’ve provided example of outdated IS in a VM SQL01 above. Cluster validation report generates a warning. PowerShell shows 0 in IntegrationServicesVersion

By default, PowerShell loads Hyper-V module with version 2. I tried to remove that module, import one with 1.1 version and query again IS versions. In that case, I received actual IS versions and  “Update required”  in IntegrationServicesState. It seems like something wrong with Hyper-V 2.0 module. Waiting for official fixes or confirmations.

hyperv

Update: this behavior is normal. VM support is based on the having critical updates and reporting IS version is no longer needed.