Using TOX_PACKAGE in CI/CD flow (rewrite/4.0)

[alexlawriewood]

Hi all,

I'm trying to tidy up our tox-centric CI/CD flow, which we re-use for a bunch of small projects. I've been enjoying playing with the 4.0 alphas (on Windows), and have been running tests with package = wheel, such that a wheel is built and its location is injected into the environment as the variable TOX_PACKAGE within the tests.

Now, the flow that I'd like to implement to take advantage of the new stuff is:

• Build wheel in tox (fine)
• Run tests in all environments (fine)
• Publish wheel via twine if tests pass

This last point isn't going quite as smoothly. There are a couple of things here: firstly, I can't figure out how the TOX_PACKAGE environment variable is intended to be used. I can't (again, on Windows) use it in a command like python -m twine %TOX_PACKAGE% because execute.local_sub_process.__init__.LocalSubProcessExecuteInstance.__enter__ calls Popen without shell=True, such that environment variable expansion isn't done.

I can't write python -m twine {env:TOX_PACKAGE} instead, because expansion of variables in the configuration is done before the package has even been built. A placeholder thing like python -c "import os; print(os.environ['TOX_PACKAGE'])" does work, so at the moment I'm invoking another cmd instance and running a batch file wrapper of twine, but I'd hope there's meant to be a better way.

Secondly (apologies if I'm jumping the gun a bit, I know the rewrite is still ongoing!), there's doesn't seem to be any mechanism for running commands at the very end of a tox execution. A hook for plugins to run at this point would seem like it might be useful. In my case, all our projects produce light and universal wheels, so I'd like to be able to do the publishing as a last step once tests pass in all environments/interpreters, and not just put it in commands_post and have it published at the end of each individual set of tests.

It seems like this would be a fairly common flow, especially with the addition of wheel-building, but maybe it's just me? It seems vaguely like this kind of thing could be a plugin (which I'd be happy to write), but the hook for running stuff at the end of all tests isn't there. Or is there a better way? I'd be interested in contributing if this is an idea that's come up before that there just hasn't been time for.

Cheers,
Alex

[gaborbernat]

Hi, your observations are correct. I personally usually defined a publish environment that used the build project to build another wheel, and this runs at the end of the other environments, mostly inline with how our CI works, see https://github.com/tox-dev/tox/blob/rewrite/.github/workflows/check.yml#L104-L125, for instance https://github.com/tox-dev/tox/actions/runs/1652936756.

A placeholder thing like python -c "import os; print(os.environ['TOX_PACKAGE'])" does work, so at the moment I'm invoking another cmd instance and running a batch file wrapper of twine, but I'd hope there's meant to be a better way.

Yes, you could do here a subprocess call invoking twine and passing the TOX_PACKAGE to it; there are no need for a batch file wrapper. (python -c "import subprocess, os; subprocess.call(['twine', 'upload', os.environ['TOX_PACKAGE']])"). That's how it is meant to be used. Shell expansion is not cross-platform compatible, hence why we don't recommend that. The package is built after the configuration is materialized so we can't inject the package name upfront via the env substation.

Secondly (apologies if I'm jumping the gun a bit, I know the rewrite is still ongoing!), there's doesn't seem to be any mechanism for running commands at the very end of a tox execution.

We're waiting for good use cases where this is needed to add it. In your case, I feel like what you really would want here is a new tox environment with skip install true that requires twine, and then make that to depends on all your other tox environments.

[alexlawriewood]

Aha, I didn't think to check your own CI! Thanks for the background about the TOX_PACKAGE usage, and for the suggestions.