VMs created with EFI missing the chipset.motherboardLayout setting

[realroywalker]

Overview of the Issue

When creating a VM (vCenter / ESX 8.0 Update 3) using the vSphere ISO method, the resulting VM works fine but does not contain the option 'chipset.motherboardLayout' when inspecting the VMX file for the VM.
If a VM is created using the vSphere UI (with the same hardware configuration) this line IS present in the file (it is set to 'acpi').

The line being missing causes problems when exporting and then re-importing the VM from a vCenter, since when it's imported the missing line gets added - the result of this is that the PCI IDs for the VM change, so internal OS IDs for things like the network interfaces change. For example in my Linux VM 'ens192' becomes 'ens33'.

The issue of changing PCI IDs is detailed here; https://knowledge.broadcom.com/external/article/313466/virtual-device-pci-slot-changes-upon-upg.html which led me to inspect that property - but even though my Packer built VM is being created on hardware version 21 it is not present. I also confirmed the workaround they list, which does solve the issue but requires manual intervention to the VMX.
I tried to use the 'configuration_parameters' option of the Packer plugin to add this option and set it to 'acpi' so that my PCI IDs remain the same regardless of whether the VM has been exported / re-imported, but the option doesn't appear to get set.

Reproduction Steps

Deploy a VM using vSphere ISO (I'm using RHEL9 not that should matter) - configuration is simply setting basic CPU, RAM etc.. nothing to specify the hardware version, so it defaults to the latest (21 in my case) and use EFI.

• Once the VM is built, power it off and inspect the advanced properties - note the ethernet0 PCI ID.
• Check the VMX file for the VM, note the lack of any line setting 'chipset.motherboardLayout'
• Export the VM to OVF and then re-import it back into vCenter
• Inspect the advanced properties again, note the ethernet0 PCI ID (it should be the same)
• Power on the imported VM
• Inspect the ethernet0 PCI ID once more.. it should now have changed!
• Check the VMX file for the imported VM, you should find a line setting 'chipset.motherboardLayout'

Packer Version

• 1.11.2

Plugin Version and Builders

• 1.4.0

Please select the builder.

• vsphere-iso
• vsphere-clone

VMware vSphere Version

• 8.0 Update 3

Guest Operating System

• RHEL 9
• Server 2022

Simplified Packer Buildfile

Unfortunately I cannot access the buildfile - the build config is noted above, it is nothing special just a simple install from ISO.

Operating System and Environment Details

• Operating System: RHEL 9

[tenthirtyam]

Can you confirm the same but without the export to OVF and reimport - does it occur with just using the template after being build and then cloned for a new VM?

[tenthirtyam]

Can you also confirm that if you build a VM manually (not using Packer) and perform the same operation steps that there is a different outcome?

[tenthirtyam]

Some additional reference:

https://williamlam.com/2023/01/acpi-motherboard-layout-requires-efi-considerations-for-switching-vm-firmware-in-vsphere-8.html

[realroywalker]

Can you confirm the same but without the export to OVF and reimport - does it occur with just using the template after being build and then cloned for a new VM?

I have tested this today - the behaviour is slightly different with no export.

• I built the VM using packer, powered it off and noted the properties inside the VMs VMX file.
  For the packer built VM, 'chipset.motherboardLayout' was missing, and the ethernet.pciSlotNumber was 160.
• I cloned the packer VM via the vSphere UI (Right Click > Clone to Virtual Machine) and inspected the VMX before powering
  it on.
  The ethernet.pciSlotNumber was still 160, but chipset.motherboardLayout had been added and was set to 'i440bx'.
• I powered up the new VM, turned it off and checked the properties again - they remained the same. So in this case the
  PCI id doesn't change

Can you also confirm that if you build a VM manually (not using Packer) and perform the same operation steps that there is a different outcome?

I carried out the same tests as above, but this time using a manually built VM (via the vSphere UI).

• The manually built VM had 'chipset.motherboardLayout' in the VMX right off the bat (it was set to 'acpi), and the ethernet.pciSlotNumber was 33.
• Then I cloned this VM using the vSphere UI and checked the properties of the new VM, they were identical (and also after powering up / back down).

So it seems that the first part of the issue is packer doesn't add the 'chipset.motherboardLayout' property for whatever reason - then because it's missing vCenter adds the property as 'acpi' (I guess it's default) when the VM is being imported. This triggers the PCI ID changes, since the board layout is different.

Quite how it manages to realise the original layout is actually i440bx when doing a clone inside vCenter I don't know.. but in that case because it's the same as the original layout the IDs don't change.

I've spun up an environment where I can test this issue easily now so feel free to let me know if you need any further things checked.

[tenthirtyam]

Curious, if you create a VM with govc, what is the default?

govc vm.create -m 2048 -c 2 -g otherGuest64 -net "your-network" -ds "your-datastore" -folder "/path/to/your/folder" -pool "your-resource-pool" your-vm-name

Reason I'm asking is I want to rule out if this is an govmomi SDK issue first.

[realroywalker]

Curious, if you create a VM with govc, what is the default?

govc vm.create -m 2048 -c 2 -g otherGuest64 -net "your-network" -ds "your-datastore" -folder "/path/to/your/folder" -pool "your-resource-pool" your-vm-name

Reason I'm asking is I want to rule out if this is an govmomi SDK issue first.

I just tried with the latest github release of govc (0.44.1) and the command as you suggested does not add the 'chipset.motherboardLayout' option to the VMX of the new VM.
However - if I supply the option "-firmware efi", then it does create with the chipset.motherboardLayout option (set to acpi).
I also tried with "-firmware bios" but this oddly didn't result in the option appearing with 'i440bx' as the setting.. it was still absent entirely.

To be clear, in my packer tests my hcl also defines the firmware as efi - so in that case I'd have expected the packer VM to have the chipset option set to acpi.

[tenthirtyam]

Thanks for confirming.

I'll take a look at this for the v1.5.0 milestone.

v1.4.1 is locked in.

[tenthirtyam]

Could you try the following as a workaround?

configuration_parameters = {
    "chipset.motherboardLayout" = "acpi"
}

[realroywalker]

Could you try the following as a workaround?

configuration_parameters = {
    "chipset.motherboardLayout" = "acpi"
}

Already tried that and no difference I'm afraid 😣