port perl based tests to python

[qknight]

Tests in tags/20.09 NixOS are now using python instead of perl. Now we need to adapt nixcloud-webservices to this change:

• https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests (documentation)
• NixOS/nixpkgs@3a28fefe (implementation)
• NixOS/nixpkgs@7e25be92 (perl 2 python test adaption)

[qknight]

I've setup several virtualbox machines: 19.09, 20.03, 20.09 and I'm facing these problems:

• SVM is not detected by virtualbox or windows, so no nested virtualization
• 19.09
  • building tests.etcd using nix-build /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/release.nix -A tests.etcd with a VM build and the tests uses perl (not python) -> shows that VM tests work in general, slow, but they work!
  • i evaluated many nixcloud-webservice tests to show they work: https://github.com/nixcloud/nixcloud-webservices/wiki/19.09-release-roadmap, some need hardware supported virtualization, SVM in my case, to work. i can't get SVM to work currently, maybe have to update the bios. (using windows 10 as host).
• 20.03
  • the settings of this virtualbox VM are similar to my 19.09 one, but they python based VM tests don't work
  • building tests.etcd using nix-build -I nixpkgs=../nixpkgs ../nixpkgs/nixos/release.nix -A tests.etcd results in:

post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver
strip is /nix/store/nyhj00w339gk2gaj3faz70gjrnbmam8v-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver/bin
patching script interpreter paths in /nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver
/nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver/bin/.nixos-test-driver-wrapped: interpreter directive changed from " /somewhere/python3" to "/nix/store/36wzi6j49g4z2qclv104a0j4vc9rakz3-python3-3.7.6-env/bin/python3"
checking for references to /build/ in /nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver...
building '/nix/store/z096bbwdbpcfbvahzmlf5svmp3dgq5bv-nixos-test-driver-etcd.drv'...
All done! ✨ 🍰 ✨
1 file would be left unchanged.
building '/nix/store/wp0jfmz8ssr8phjyy1fxvsinndr1nmr2-vm-test-run-etcd.drv'...
starting VDE switch for network 1
running the VM test script
should start etcd node
node: starting vm
node # Formatting '/build/vm-state-node/node.qcow2', fmt=qcow2 size=536870912 cluster_size=65536 lazy_refcounts=off refcount_bits=16
node: QEMU running (pid 9)
node: waiting for the VM to finish booting
node # qemu-system-x86_64: CPU model 'host' requires KVM
node: connected to guest root shell
node: (connecting took 0.04 seconds)
(0.04 seconds)
error: [Errno 32] Broken pipe
(0.86 seconds)
should write and read some values to etcd
node: must succeed: etcdctl set /foo/bar 'Hello world'
error: [Errno 32] Broken pipe
(0.00 seconds)
(0.86 seconds)
Traceback (most recent call last):
  File "/nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver/bin/.nixos-test-driver-wrapped", line 944, in <module>
    run_tests()
  File "/nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver/bin/.nixos-test-driver-wrapped", line 879, in run_tests
    machine.execute("sync")
  File "/nix/store/n5np4bzjb51y2fpz033d3nn83girm65f-nixos-test-driver/bin/.nixos-test-driver-wrapped", line 393, in execute
    self.shell.send(out_command.encode())
BrokenPipeError: [Errno 32] Broken pipe
cleaning up
killing node (pid 9)
(0.00 seconds)

Maybe realted to NixOS/nixpkgs#147294 but the sleep fix did not work for me.

Update on 20.09 python vs. perl tests (windows virtualbox vm)

Checkout to 19.09 etcd.nix with perl also results in an VM not working:

error: the VM quit before connecting

full log

building '/nix/store/mg4v2zp2qk88dddpls4lyaiic3vw73m8-nixos-test-driver-etcd.drv'...
building '/nix/store/gjx0c55ipca3s0pmi8z0pf0v5d73w5lb-vm-test-run-etcd.drv'...
starting VDE switch for network 1
running the VM test script
subtest: should start etcd node
node: starting vm
node# Formatting '/build/vm-state-node/node.qcow2', fmt=qcow2 size=536870912 cluster_size=65536 lazy_refcounts=off refcount_bits=16
node: QEMU running (pid 9)
node: waiting for unit ‘etcd.service’
node: running command: systemctl --no-pager show "etcd.service"
node: waiting for the VM to finish booting
node# qemu-system-x86_64: CPU model 'host' requires KVM
(0.02 seconds)
(0.02 seconds)
(0.02 seconds)
error: the VM quit before connecting
(0.15 seconds)
subtest: should write and read some values to etcd
node: must succeed: etcdctl set /foo/bar 'Hello world'
node: waiting for the VM to finish booting
(0.00 seconds)
(0.00 seconds)
error: the VM quit before connecting
(0.00 seconds)
(0.15 seconds)
collecting coverage data
(0.00 seconds)
syncing
(0.00 seconds)
0 out of 2 tests succeeded
test script finished in 0.15s
cleaning up
killing node (pid 9)
(0.00 seconds)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/gjx0c55ipca3s0pmi8z0pf0v5d73w5lb-vm-test-run-etcd.drv' failed with exit code 1

update 20.09 (thinkpad, nixos as bases system with nested virtualization working)

just works. turns out that 20.09 requires nested virtualization or the tests won't execute at all.

[qknight]

I will port this in 20.03 because there the perl tests still work, so I only need to fix one implementation.

[qknight]

major progress in branch: nixpkgs-20.03-tests-perl2python-port