Roundtripping a commit via oci image looses the commitmeta which makes signed ostree break

[alexlarsson]

This was moved here from coreos/rpm-ostree#4973

I created an oci image based on an automotive image that used signed composefs, when I rebased to it everything works, but on reboot the signatures fails:

[ 0.419101] ostree-prepare-root[528]: ostree-prepare-root: Error loading signatures from repo: No commitmeta for commit 24e46035399c03969d5e6f610425a9adfc7a9959507fe9c32f6ccd12b3467573

Seems the detached metadata is lost somewhere along the way, which means that the signature is not available in the deployed repo.

[cgwalters]

This is a confusing topic, and I would probably agree this is a bug, but it was intentional in the code today.

There are two different codepaths. Basically this (should) work today when using ostree container encapsulate and ostree container unencapsulate.

When following that path, the container metadata is discarded (manifest, config) - there is nothing else stored except the ostree commit. The fact that there was a container image involved in transport is just discarded.

But when using ostree container image (a distinct layer in this code) and specifically the ostree container image pull/deploy functionality which is used by bootc (and rpm-ostree's container bits), we e.g. actually retain distinct refs for each container image layer (the ostree/container namespace) and we also create a merge commit.

The thing is, when using the container flow today we currently default to expecting to handle derived layers.

We used to have code which tried to split the difference here...and arguably, it is a bug or at least a design misfeature that we do a merge commit in this case, breaking any ostree signatures at deployment time.

This issue touches on #388 too - if we encourage people to do that, we can start to lean on using ostree signatures in encapsulated container images.

However...IMO, the direction we want to go is less around making ostree-containers special; we should create an opinionated binding between signed composefs and generic OCI. I will file something about this...

[cgwalters]

In the short term...I think we can change the store.rs code to go back to not creating a merge commit, but then we'll need to store the manifest and config $somewhere else. (This isn't hard of course, we could stick them in a lookaside somewhere like /ostree/repo/ext/container or whatever, or maybe better special refs like ostree/container/meta...the downside is just that we'd "leak space" if one ever did a transition from container -> ostree and stopped using the ostree-container stack. But maybe we should stop caring about that...

[cgwalters]

This issue (among others) made me feel the need to do a spike on really rethinking how we do a sane container storage, which led to containers/composefs#286 FWIW

[cgwalters]

Also cc containers/bootc#20 and https://github.com/cgwalters/composefs-oci