Installing multiple nested packages

[rmartin16]

I've been trying to work out how tox can install multiple packages for a project.

This is a simplified version of the src layout.

project
├── core
│   ├── pyproject.toml
│   ├── README.rst
│   ├── src
│   │   ├── pkg1
│   └── tests
├── docs
├── dummy
│   ├── pyproject.toml
│   ├── README.rst
│   └── src
│       ├── pkg2
├── LICENSE
├── pyproject.toml
├── README.rst
├── tox.ini

Can tox be configured to install both the core and dummy packages?

Currently, I'm running pip install ./core ./dummy as one of the commands but this creates issues for running envs in parallel because multiple envs are trying to build and install the packages. I think this could be resolved if tox can create wheels and install them for each env.

[gaborbernat]

tox today supports a single package. You can create a plugin that supports multiple packages, but we don't plan to support it as part of the core for now. As you see here https://github.com/tox-dev/tox/blob/main/src/tox/tox_env/python/runner.py#L120C27-L120C68 we do allow for multiple packages, but the default packager here

tox/src/tox/tox_env/python/virtual_env/package/pyproject.py

Lines 211 to 267 in 0fd872d

	def perform_packaging(self, for_env: EnvConfigSet) -> list[Package]:
	"""Build the package to install."""
	try:
	deps = self._load_deps(for_env)
	except BuildEditableNotSupportedError:
	self.call_require_hooks.remove("editable")
	targets = [e for e in self.builds.pop("editable") if e["package"] == "editable"]
	names = ", ".join(sorted({t.env_name for t in targets if t.env_name}))
	logging.error( # noqa: TRY400
	"package config for %s is editable, however the build backend %s does not support PEP-660, falling "
	"back to editable-legacy - change your configuration to it",
	names,
	cast(Pep517VirtualEnvFrontend, self._frontend_).backend,
	)
	for env in targets:
	env._defined["package"].value = "editable-legacy" # type: ignore[attr-defined] # noqa: SLF001
	self.builds["editable-legacy"].append(env)
	self._run_state["setup"] = False # force setup again as we need to provision wheel to get dependencies
	deps = self._load_deps(for_env)
	of_type: str = for_env["package"]
	if of_type == "editable-legacy":
	self.setup()
	config_settings: ConfigSettings = self.conf["config_settings_get_requires_for_build_sdist"]
	sdist_requires = self._frontend.get_requires_for_build_sdist(config_settings=config_settings).requires
	deps = [*self.requires(), *sdist_requires, *deps]
	package: Package = EditableLegacyPackage(self.core["tox_root"], deps) # the folder itself is the package
	elif of_type == "sdist":
	self.setup()
	with self._pkg_lock:
	config_settings = self.conf["config_settings_build_sdist"]
	sdist = self._frontend.build_sdist(sdist_directory=self.pkg_dir, config_settings=config_settings).sdist
	sdist = create_session_view(sdist, self._package_temp_path)
	self._package_paths.add(sdist)
	package = SdistPackage(sdist, deps)
	elif of_type in {"wheel", "editable"}:
	w_env = self._wheel_build_envs.get(for_env["wheel_build_env"])
	if w_env is not None and w_env is not self:
	with w_env.display_context(self._has_display_suspended):
	return w_env.perform_packaging(for_env)
	else:
	self.setup()
	method = "build_editable" if of_type == "editable" else "build_wheel"
	config_settings = self.conf[f"config_settings_{method}"]
	with self._pkg_lock:
	wheel = getattr(self._frontend, method)(
	wheel_directory=self.pkg_dir,
	metadata_directory=self.meta_folder_if_populated,
	config_settings=config_settings,
	).wheel
	wheel = create_session_view(wheel, self._package_temp_path)
	self._package_paths.add(wheel)
	package = (EditablePackage if of_type == "editable" else WheelPackage)(wheel, deps)
	else: # pragma: no cover # for when we introduce new packaging types and don't implement
	msg = f"cannot handle package type {of_type}"
	raise TypeError(msg) # pragma: no cover
	return [package]

only supports one.

[gaborbernat]

Should you create that plugin, please post it here 👍 for awareness of other people.