Add vTPM test #1011

shjala · 2024-08-13T15:32:50Z

TestVtpmIsRunningOnEVE checks if the vTPM process is running on the EVE node, it does this by checking if the vTPM control socket is open and the vTPM process is listening on it.
TestVtpmIsStatePreservation checks if the vTPM state is preserved after a reboot, it does this by creating a key in the vTPM (through a VM running on EVE) and then rebooting the EVE node, after the reboot it checks if the key is still present in the vTPM, by getting the list of vTPM persistent keys (through the the VM running on EVE).
TestAzureIotTPMEndrolmentWithVTPM tests the end-to-end scenario of enrolling a TPM device in Azure IoT Hub, this test will create a TPM enrollment in Azure IoT Hub, configure the Azure IoT Edge in a VM running on EVE, and check if the services are running.

This tests lf-edge/eve#4071

Wait for next release with Qemu : add patch to fix bios linker loader assert eve#4204
Open a PR to set eve tag to the latest
Rebase against latest master
Reduce the test verbosity before merging

/cc @eriknordmark

shjala · 2024-08-15T10:56:43Z

I'll fix the yetus errors and squash at the end.

shjala · 2024-08-15T11:37:33Z

should go after #1012 and #1013

shjala · 2024-08-16T12:39:47Z

Running it locally with secrets available works fine and test pass, and when I run it using ./eden test tests/workflow/ -s smoke.tests.txt it fails as expected, complaining about the environment variables not being set :

--- FAIL: TestEdenScripts (0.00s)
    --- FAIL: TestEdenScripts/tpm_eden (88.65s)
        testscript.go:418:
            > test eden.aziot.test
            testProg: /home/shah/shah-dev/eden/dist/bin/eden.aziot.test
            [stdout]
            configName:  default
            configFile:  /home/shah/.eden/contexts/default.yml
            time="2024-08-16T15:34:01+03:00" level=info msg="TestAzureIotTPMEndrolment with EVE-Tools and Proxy TPM started"
            configName:  default
            configFile:  /home/shah/.eden/contexts/default.yml
            time="2024-08-16T15:34:01+03:00" level=debug msg="Will use config from /home/shah/.eden/contexts/default.yml"
            time="2024-08-16T15:34:01+03:00" level=debug msg="Try to add config from $WORK/eden-config.yml"
            configName:  default
            configFile:  /home/shah/.eden/contexts/default.yml
            time="2024-08-16T15:34:01+03:00" level=debug msg="Will use config from /home/shah/.eden/contexts/default.yml"
            time="2024-08-16T15:34:01+03:00" level=debug msg="Try to add config from $WORK/eden-config.yml"
            time="2024-08-16T15:34:01+03:00" level=debug msg="Will use config from /home/shah/.eden/contexts/default.yml"
            time="2024-08-16T15:34:01+03:00" level=debug msg="Try to add config from $WORK/eden-config.yml"
            time="2024-08-16T15:34:01+03:00" level=debug msg="new datastore created 1d1bff96-a0b6-4e22-bd37-708b1a703e43"
            time="2024-08-16T15:34:01+03:00" level=info msg="Starting download of image from https://cloud-images.ubuntu.com/releases/20.04/release/ubuntu-20.04-server-cloudimg-amd64.img"
            time="2024-08-16T15:34:01+03:00" level=info msg="Start download into eserver of ubuntu-20.04-server-cloudimg-amd64.img"
            time="2024-08-16T15:34:01+03:00" level=info msg="Downloading... Ready 0 B"
            time="2024-08-16T15:34:06+03:00" level=info msg="Downloading... Ready 214 MB"
            time="2024-08-16T15:34:11+03:00" level=info msg="Downloading... Ready 465 MB"
            time="2024-08-16T15:34:16+03:00" level=info msg="Image downloaded with size 627 MB and sha256 256c73e2d77808f834c66a5c40f4e97f91a70a6e6ce7424bc91b54b36aceadec"
            time="2024-08-16T15:34:16+03:00" level=debug msg="new image created bf00e810-311c-46cb-a971-14c8bee3fc85"
            config changed, to see config run 'eden controller edge-node get-config'
            time="2024-08-16T15:34:16+03:00" level=debug msg="VersionIncrement 5->6"
            time="2024-08-16T15:34:17+03:00" level=debug msg="config updated /home/shah/.eden/devUUID-8fd9ee4f-a9d1-465f-aa22-7f72d234963e.json"
            time="2024-08-16T15:34:17+03:00" level=debug msg="state updated /home/shah/.eden/state-33c919bb-af75-477f-afbc-8f10eadbfca1.yml"
            time="2024-08-16T15:34:17+03:00" level=info msg="deploy pod aziot-quizzical_curie with https://cloud-images.ubuntu.com/releases/20.04/release/ubuntu-20.04-server-cloudimg-amd64.img request sent"
            time="2024-08-16T15:34:47+03:00" level=debug msg="will use remote adam loader"
            time="2024-08-16T15:34:47+03:00" level=debug msg="NewRedisLoader init"
            time="2024-08-16T15:34:47+03:00" level=debug msg="XRead from INFO_EVE_8fd9ee4f-a9d1-465f-aa22-7f72d234963e"
            time="2024-08-16T15:34:47+03:00" level=debug msg="will use remote adam loader"
            time="2024-08-16T15:34:47+03:00" level=debug msg="NewRedisLoader init"
            time="2024-08-16T15:34:47+03:00" level=debug msg="XRead from METRICS_EVE_8fd9ee4f-a9d1-465f-aa22-7f72d234963e"
            time="2024-08-16T15:35:27+03:00" level=info msg="TestAzureIotTPMEndrolment finished"
            --- FAIL: TestAzureIotTPMEndrolmentWithEveTools (86.45s)
                aziot_test.go:90: AZIOT_CONNECTION_STRING environment variable is not set
            FAIL
            config changed, to see config run 'eden controller edge-node get-config'
            time="2024-08-16T15:35:27+03:00" level=debug msg="VersionIncrement 6->7"
            time="2024-08-16T15:35:28+03:00" level=debug msg="config updated /home/shah/.eden/devUUID-8fd9ee4f-a9d1-465f-aa22-7f72d234963e.json"
            time="2024-08-16T15:35:28+03:00" level=debug msg="state updated /home/shah/.eden/state-33c919bb-af75-477f-afbc-8f10eadbfca1.yml"
            time="2024-08-16T15:35:28+03:00" level=info msg="app aziot-quizzical_curie stop done"
            config changed, to see config run 'eden controller edge-node get-config'
            time="2024-08-16T15:35:28+03:00" level=debug msg="VersionIncrement 7->8"
            time="2024-08-16T15:35:29+03:00" level=debug msg="config updated /home/shah/.eden/devUUID-8fd9ee4f-a9d1-465f-aa22-7f72d234963e.json"
            time="2024-08-16T15:35:29+03:00" level=debug msg="state updated /home/shah/.eden/state-33c919bb-af75-477f-afbc-8f10eadbfca1.yml"
            time="2024-08-16T15:35:29+03:00" level=info msg="app aziot-quizzical_curie delete done"
            time="2024-08-16T15:35:29+03:00" level=info msg="Azure IOT Hub Test finished"

but here it time out because the app is not available ?!

shjala · 2024-08-23T06:49:14Z

this is rebased against master and #1016 , clean it up after #1017 is merged.

uncleDecart · 2024-08-26T11:35:58Z

@shjala can you rebase on master since I just merged #1019, let's see if the test works

uncleDecart · 2024-08-26T11:37:04Z

Also there is this PR #1008 which bumps EVE version to 13.0.0, can we bump EVE version to 13.0.0 in your PR as well?

uncleDecart

Do I get it right, that the scripts we have are the ones running inside VMs?

uncleDecart · 2024-08-26T11:37:30Z

go.mod

-go 1.20
+go 1.22
+
+toolchain go1.22.5


why do we need toolchain?

Ahhhh, I was to late to comment on toolchain :DD

uncleDecart · 2024-08-26T11:38:36Z

tests/aziot/aziot_test.go

+	// The ID Scope is required to configure azure-iot in the VM,
+	// we can get it from the Azure IoT Hub -> Device Provisioning Service -> Overview
+	// and copy the "ID Scope".
+	aziotIdScope = os.Getenv("AZIOT_ID_SCOPE")


Yetus complaint to rename variable :)

uncleDecart · 2024-08-26T11:39:45Z

tests/aziot/aziot_test.go

+	}()
+
+	// wait for the deployed app to appear in the list
+	time.Sleep(30 * time.Second)


wasn't there a way to read app status directly from controller?

yes, but in the first few seconds it return "app not found", even though it is deployed.

see a few lines down :

err = eveNode.AppWaitForRunningState(appName, 60*5) if err != nil { t.Fatalf("Failed to wait for app to start: %v", err) }

huh, so that's a bug and a behaviour we don't want in Eden, I'll create a ticket for it then. Because it shouldn't be like that, right? :D

I think so, let me check again and make sure it is happening consistently or no.

uncleDecart · 2024-08-26T11:44:45Z

tests/aziot/aziotenroll.go

+}
+
+func readPublicKey(handle tpmutil.Handle) ([]byte, error) {
+	// unfortunaly we can't used SWTPM socket directly, it is blocked becuse


Minor yetus changes

will fix all/squash after the review is completed.

shjala · 2024-08-26T11:57:42Z

Do I get it right, that the scripts we have are the ones running inside VMs?

yes, the scripts are copied to vm to set up the azure-iot-edge and run the services, then I check if everything is running without error.

pkg/evetestkit/utils.go

uncleDecart · 2024-08-26T12:10:12Z

tests/aziot/aziot_test.go

+		}
+
+		// execute the script to create the necessary TPM keys
+		_, err = eveNode.AppSSHExec(appName, createKeyScriptPath)


side note: I'd abstract path to a file into a reader for AppSSHExec function, it'll be more flexible to call and if you want to run some local examples (unit tests for a function) you can create readers in code :)

Edit: I meant AppSCPCopy, SSHExec runs command AFAIK

you mean change AppSCPCopy to something like AppSCPCopy(strAppname, ioReader, strRemoteFile) ?
but the internal scp call needs a path.

This is a test for EVE PR #4071. It test the vTPM feature of EVE and aziot on EVE both legacy (using EVE-TOOLS) and latest (using vTPM). Signed-off-by: Shahriyar Jalayeri <[email protected]>

shjala · 2024-10-11T14:04:53Z

@yash-zededa the variables are still not accessible in the test, your help is much appreciated.

time="2024-09-16T13:06:44Z" level=fatal msg="AZIOT_CONNECTION_STRING environment variable is not set"

yash-zededa · 2024-10-11T14:24:55Z

@yash-zededa the variables are still not accessible in the test, your help is much appreciated.
time="2024-09-16T13:06:44Z" level=fatal msg="AZIOT_CONNECTION_STRING environment variable is not set"

PR's won't have access to secrets. Unless WF uses pull_request_target

shjala · 2024-10-14T07:47:37Z

@yash-zededa the variables are still not accessible in the test, your help is much appreciated.
time="2024-09-16T13:06:44Z" level=fatal msg="AZIOT_CONNECTION_STRING environment variable is not set"
PR's won't have access to secrets. Unless WF uses pull_request_target

You mean there is no way to test this is working without merging?

yash-zededa · 2024-10-14T08:23:24Z

@yash-zededa the variables are still not accessible in the test, your help is much appreciated.
time="2024-09-16T13:06:44Z" level=fatal msg="AZIOT_CONNECTION_STRING environment variable is not set"
PR's won't have access to secrets. Unless WF uses pull_request_target
You mean there is no way to test this is working without merging?

Unfortunately, yes, this behavior is intentional from GitHub. If you want to test pull requests with logins, the workflow needs to be updated to trigger using pull_request_target, which will give the actions access to secrets for testing.

However, this approach allows anyone to modify the workflow and potentially access the secrets, so it's safer to prevent pull requests from triggering if there are any changes to the workflow.

shjala requested a review from uncleDecart as a code owner August 13, 2024 15:32

shjala marked this pull request as draft August 13, 2024 15:33

shjala force-pushed the add_aziot_test branch from a2c656c to 8e4889f Compare August 14, 2024 15:58

shjala mentioned this pull request Aug 14, 2024

Full Virtual TPM for VMs and Containers lf-edge/eve#4071

Merged

13 tasks

shjala force-pushed the add_aziot_test branch 2 times, most recently from 94d8727 to b7039c7 Compare August 15, 2024 10:44

shjala changed the title ~~[wip] add aziot test~~ Add aziot test Aug 15, 2024

shjala changed the title ~~Add aziot test~~ Add azure iot edge test Aug 15, 2024

shjala force-pushed the add_aziot_test branch from b7039c7 to 5deeed0 Compare August 15, 2024 10:46

shjala marked this pull request as ready for review August 15, 2024 10:47

shjala force-pushed the add_aziot_test branch from 5deeed0 to 1e6edb1 Compare August 15, 2024 10:56

shjala force-pushed the add_aziot_test branch from 1e6edb1 to ebcbb9a Compare August 15, 2024 11:07

shjala force-pushed the add_aziot_test branch 6 times, most recently from 826d976 to cc8039d Compare August 16, 2024 11:38

shjala marked this pull request as draft August 22, 2024 11:48

shjala force-pushed the add_aziot_test branch 2 times, most recently from c48ff14 to 9132daf Compare August 23, 2024 06:14

shjala marked this pull request as ready for review August 23, 2024 06:19

shjala changed the title ~~Add azure iot edge test~~ [WIP] Add azure iot edge test Aug 23, 2024

shjala force-pushed the add_aziot_test branch from 612a548 to f811ef7 Compare August 23, 2024 06:20

shjala mentioned this pull request Aug 26, 2024

add secrets for aziot test #1018

Closed

shjala force-pushed the add_aziot_test branch from df34ff3 to 5abe22d Compare August 26, 2024 09:17

yash-zededa mentioned this pull request Aug 26, 2024

pass azure IOT secrets to composite actions via the inputs #1019

Merged

uncleDecart reviewed Aug 26, 2024

View reviewed changes

shjala force-pushed the add_aziot_test branch from 13c6f0f to d9ecfb6 Compare August 26, 2024 11:46

uncleDecart reviewed Aug 26, 2024

View reviewed changes

pkg/evetestkit/utils.go Outdated Show resolved Hide resolved

uncleDecart reviewed Aug 26, 2024

View reviewed changes

shjala marked this pull request as draft August 28, 2024 11:01

shjala force-pushed the add_aziot_test branch 2 times, most recently from a358341 to ac86cf9 Compare September 4, 2024 09:02

shjala self-assigned this Sep 4, 2024

shjala changed the title ~~[WIP] Add azure iot edge test~~ Add azure iot edge test Sep 4, 2024

shjala changed the title ~~Add azure iot edge test~~ Add azure-iot-edge test Sep 4, 2024

shjala force-pushed the add_aziot_test branch 3 times, most recently from 4f46f8a to 6a6e8e8 Compare September 11, 2024 11:16

shjala changed the title ~~Add azure-iot-edge test~~ Add vTPM test Sep 11, 2024

shjala force-pushed the add_aziot_test branch from 6a6e8e8 to 63dc2a3 Compare September 11, 2024 12:00

Add test for vTPM

279f8aa

This is a test for EVE PR #4071. It test the vTPM feature of EVE and aziot on EVE both legacy (using EVE-TOOLS) and latest (using vTPM). Signed-off-by: Shahriyar Jalayeri <[email protected]>

shjala force-pushed the add_aziot_test branch from 63dc2a3 to 279f8aa Compare October 11, 2024 14:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vTPM test #1011

Add vTPM test #1011

shjala commented Aug 13, 2024 •

edited

Loading

shjala commented Aug 15, 2024 •

edited

Loading

shjala commented Aug 15, 2024 •

edited

Loading

shjala commented Aug 16, 2024 •

edited

Loading

shjala commented Aug 23, 2024

uncleDecart commented Aug 26, 2024

uncleDecart commented Aug 26, 2024

uncleDecart left a comment

uncleDecart Aug 26, 2024

uncleDecart Aug 26, 2024

uncleDecart Aug 26, 2024

uncleDecart Aug 26, 2024

shjala Aug 26, 2024

shjala Aug 26, 2024

uncleDecart Aug 26, 2024

shjala Aug 28, 2024

uncleDecart Aug 26, 2024

shjala Aug 26, 2024

shjala commented Aug 26, 2024

uncleDecart Aug 26, 2024 •

edited

Loading

shjala Aug 28, 2024

shjala commented Oct 11, 2024

yash-zededa commented Oct 11, 2024

shjala commented Oct 14, 2024

yash-zededa commented Oct 14, 2024

Add vTPM test #1011

Are you sure you want to change the base?

Add vTPM test #1011

Conversation

shjala commented Aug 13, 2024 • edited Loading

shjala commented Aug 15, 2024 • edited Loading

shjala commented Aug 15, 2024 • edited Loading

shjala commented Aug 16, 2024 • edited Loading

shjala commented Aug 23, 2024

uncleDecart commented Aug 26, 2024

uncleDecart commented Aug 26, 2024

uncleDecart left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shjala commented Aug 26, 2024

uncleDecart Aug 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shjala commented Oct 11, 2024

yash-zededa commented Oct 11, 2024

shjala commented Oct 14, 2024

yash-zededa commented Oct 14, 2024

shjala commented Aug 13, 2024 •

edited

Loading

shjala commented Aug 15, 2024 •

edited

Loading

shjala commented Aug 15, 2024 •

edited

Loading

shjala commented Aug 16, 2024 •

edited

Loading

uncleDecart Aug 26, 2024 •

edited

Loading