How to customize a VMware ESXi image and install it in a Hyper-V VM


I’ve been doing recently  VMware ESXi deployment in my lab environment and would like to share main steps required to make it work on nested Hyper-V. Needless to say, nested virtualization works great only for demo and labs, therefore, running ESXi under Hyper-V is a completely unsupported in production environments.

Anyway, carry out the following steps to install ESXi (6.0, in my case. although these steps should work for newer versions as well):

1. Download VMWare ESXi offline bundle available at product download page (e.g. ESXi6.0). You can also download a ESXi image customized by vendor. For example, here is a direct download link for Dell’s ESXi 6.0 image which includes Dell’s VIBs in addition to built-in installation bundles provided by VMware.

2. Download the driver  which allows running ESXi as a VM under Microsoft Hyper-V (net-tulip, it’s actually a network driver which should be added to ESXi image. Otherwise, ESXi installation will be blocked)

3. Copy the downloaded files to the same folder (e.g. ‘D:\Images\VMware ESXi 6\’). It’ll be used as a work folder.

4. Download and install VMware PowerCLI 6.3 or newer

5. Once PowerCLI is installed,  run it and set location to the folder containing the files, then add offline depot ZIP files to the current PowerCLI session as shown below:

cd 'D:\Images\VMware ESXi 6\'
Add-EsxSoftwareDepot .\esxi60_bundle.zip
Add-EsxSoftwareDepot .\net-tulip-1.1.15-1-offline_bundle.zip

6. Retrieve the name of the standard image profile and note it (it’ll will be used as a clone for a new profile):

Get-EsxImageProfile|ft Name

7. Create a new image profile by cloning existing profile which name you just noted, and then add the driver’s package to the profile:

#Create a new image profile
New-EsxImageProfile -CloneProfile ESXi-6.0.0-2494585  -Name rlevchenko.com -Vendor custom

#Add custom packages
Add-EsxSoftwarePackage -ImageProfile rlevchenko.com -SoftwarePackage net-tulip -Force

image

If AcceptanceLevel is set to PartnerSupported by default (as in the picture above) and custom packages which you are going to add to the image profile have Community acceptance level, you will receive an error during creating an ESXi ISO and it’s installation . To resolve this, set the acceptance level of the image profile to CommunitySupported by running the following command: Set-EsxImageProfile -AcceptanceLevel CommunitySupported –ImageProfile rlevchenko.com

image

8. Now it’s time to create an ISO from the customized ESXi image.To do this, run the following command:

Export-EsxImageProfile -ImageProfile rlevchenko.com -FilePath D:\Images\esxi60_custom.iso -ExportToIso -Force

Create a new VM with the following settings:

  • Generation 1
  • Static RAM (> 4Gb is recommended)
  • More than 1vCPU
  • Legacy network adapter connected to the switch

A sample of VM’s configuration is shown below:

image

9. Once you finished to configure a VM, enable virtualization extensions on the VM’s CPU. Optionally, you can download a script available at github to check VM’s configuration and  enable nested virtualization. Both options are allowed:

#Option 1
Set-VMProcessor -VMName "vHost-01" -ExposeVirtualizationExtensions $True

#Option2
 .\Enable-NestedVM.ps1 -vmName "vHost-01"

10. Turn on the VM, attach the created ESXi ISO and press TAB on the boot screen, then type ignoreHeadless=TRUE and press Enter. Otherwise, ESXi boot will hang while booting  (I assume it’s all because ESXi is running on non-HCL hardware. VM is a bit out of the HCL list..).

esxi on hyper-v_1

11. Complete ESXi installation process (as usual), reboot it and press SHIFT+O during the startup, and then enable ignoreHeadless option again as shown in the screenshot:

esxi on hyper-v_4

Once ESXi is successfully started, define settings for management network, enable a Shell, and then press Alt+F1 to enter to a console. We need to set a VMKernel boot-time parameter. Otherwise, you will always need to enable ignoreHeadless after every reboot.

Provide root credentials and  type esxcfg-advcfg -k TRUE ignoreHeadless

esxi on hyper-v_6

Close the console by pressing ALT+F2, reboot ESXi and verify that it starts up seamlessly.

That’s it. Now you have a ESXi host running on a Hyper-V VM.

Until then,

enjoy your day :)!

16 thoughts on “How to customize a VMware ESXi image and install it in a Hyper-V VM”

    1. You are right. Net-tulip is “officially” compatible with ESXi up to 6.0 and, as far as I know, VMware black listed this driver in the latest ESXi. I hope things will be changed in near future.

      Reply

  2. Hi @rlevchenko

    Thanks for this write up. Can this work with Azure dv3 instances?

    Also the wiki for the driver states support for esxi 6.5. However the comment from evgen above says otherwise.

    Reply

    1. Yes, it should work in Azure (however, I’ve not tried..but don’t see any obstacles)
      Wiki says “Does NOT work With: ESXi 6.7 and newer” , so I’m not sure that it works with ESXi 6.5 even if it’s listed as supported. Additional verification/test is needed.

      Reply

  3. Hi @rlevchenko, thanks for the blog!
    I’ve seen and followed several websites with the same process and I downloaded the same sources and compiled the ISO using identical steps, but obviously it fails at my end…

    My system runs HyperV in Windows 2016 (LTS 1607) server and the VM is configured for ExposeVirtualizationExtensions and the legacy network adapter. The ignoreHeadless=TRUE switch is used to start the installer, but still it cannot load nfs41 during the setup loader and the system fails with the ‘no network adapter found’ error. I tried ESXi 6.0 as well as 6.5 and did the build with both the ESXi-Customizer-PS-v2.6.0.ps1 script as well as the ‘manual’ steps in the PowerCLI.. All same result. Any idea where this could go wrong?

    Thanks for any suggestion.

    Reply

  4. Hi @rlevchenko,

    thanks for the writeup. I’m following the steps 1:1 from this site as well as some others that did the same. Still on my laptop (Windows 2016 LTS 1607) with HyperV (running several other VMs incl some that use the legacy adapter), I cannot successfully install ESXi. The ISO is created and the tulip package has been added. The VM boots with the ignoreHeadless=TRUE switch, but during the phase that the modules are loading the message that nfs41`driver cannot be loaded shows. Later it fails with the no network adapters found message. I tried 6.5.0, and 6.0.0. Both fail the same way.
    When I prep an ESXi ISO with 5.5.0 the identical way (using the PowerCLI or ESXi-Customizer-PS-v2.6.0.ps1 script) that 5.5.0 ESXi installs fine. So obviously something to do with the net-tulip-1.1.15-1-offline_bundle not being fully compatible with the 6.0 and 6.5 but according to your blog and other that same config at least installs.

    Any idea(s)?
    thanks!

    Reply

    1. First of all, what’s your goal? running VMs? There are some limitations that make such configuration absolutely useless (for example, we can’t load x64 OSes on nested ESXi running on Hyper-V). Regarding your issue, can you double check vm configuration? + it looks like there is something wrong with your image… you can also try another one and repeat customization/installation steps.

      Reply

  5. Followed everything above and ESXi is installed and running however whenever I try to start a VM inside it I get an error of “Failed to power on virtual machine Test. VMware ESX and Hyper-V are not compatible. Remove the Hyper-V role from the system before running VMware ESX.”
    I used the ESXi 6.0 Update 2 offline bundle to create the ISO.

    Reply

  7. Hi Roman
    I followed your procedure with the ESXi 6.0 downloaded from your link (i.e. DELL 6.0.0-2494585 etc), and it works.
    If I download the offline bundle from vm-ware, I get “ESXi-6.0.0-20200204001-standard.zip”, and it does not work !!! (no adapter found).
    [if it does not work with 6.0, it will not work with 6.5)

    What’s wrong with my vmware offline bundle ? Is there a “threshold” version, after which NET-TULI cannot run ?

    Thank you very much !

    Reply

      1. Hi Roman. I think there is a bit of misunderstanding. Neither official nor Dell v 5.0 offline depots include net-tuli driver. Your procedure works with old v6.0 depot (Dell) but not with new v6.0 depot (Vmware). You don’t use esxi from a long time: ok, I understand. Do you know where to findcan old v6.5 depot? (To check with your procedure). Thank you. Julian

        Reply

  8. Hello , I am able to host ESXI 6.0 on hyper on Azure But facing issue while create the VM on that ESXI :ie. VMware Workstation and Hyper-V are not compatible. Remove the Hyper-V role from the system before running. Do you have Idea

    Reply

  9. Thanks for the great detailed procedure. The only issue I am facing is that the ESXi installed inside Hyper-V cannot reach via the network other VMs installed on the same Hyper-V host. It can anyway ping and be reached by other physical machines on the network. All machines are on the same virtual switch. Is it a known limitation?

    Reply

Leave a reply to rlevchenko Cancel reply