PyprojectTomlConfigSettingsSource overwrite toml_file Path from model_config

[sbrugman]

To use pydantic for handling pyproject.toml configuration, I would like to be able to pass a custom path to the PyprojectTomlConfigSettingsSource. This is possible for the TomlConfigSettingsSource source, but that lacks the pyproject.toml support (very minor layer to get the tool.my-tool section of the Toml file. For PyprojectTomlConfigSettingsSource, the settings_cls.model_config.get('toml_file') lookup is missing.

toml_file is ignored:

class Settings(BaseSettings):
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (PyprojectTomlConfigSettingsSource(settings_cls),)

class MySettings(BaseSettings):
    model_config = SettingsConfigDict(toml_file=Path("config.toml").resolve(), pyproject_toml_table_header=("tool", "my-tool"))

toml_file is respected:

class Settings(BaseSettings):
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (TomlConfigSettingsSource(settings_cls),)

class MySettings(BaseSettings):
    model_config = SettingsConfigDict(toml_file=Path("config.toml").resolve())

It looks like a fairly simple and straight-forward enhancement in sources.py.

My current workaround is to create a custom class with elements of both classes:

class CustomPyprojectTomlConfigSettingsSource(TomlConfigSettingsSource):
    def __init__(
        self,
        settings_cls: type[BaseSettings],
        toml_file: PathType | None = DEFAULT_PATH,
    ):
        self.toml_file_path = (
            toml_file
            if toml_file != DEFAULT_PATH
            else settings_cls.model_config.get(
                "toml_file",
                PyprojectTomlConfigSettingsSource._pick_pyproject_toml_file(
                    toml_file, settings_cls.model_config.get("pyproject_toml_depth", 0)
                ),
            )
        )
        self.toml_data = self._read_files(self.toml_file_path)
        self.toml_table_header: tuple[str, ...] = settings_cls.model_config.get(
            "pyproject_toml_table_header", ("tool", "pydantic-settings")
        )
        for key in self.toml_table_header:
            self.toml_data = self.toml_data.get(key, {})
        super(TomlConfigSettingsSource, self).__init__(settings_cls, self.toml_data)

Would be great if this functionality can come out-of-the-box.

[hramezani]

Thanks @sbrugman for reporting this issue. Yeah, the PyprojectTomlConfigSettingsSource doesn't consider the toml_file in the config. actually, it doesn't have any config.

BTW, I would suggest defining a new config for PyprojectTomlConfigSettingsSource because toml_file accepts list of files but PyprojectTomlConfigSettingsSource accepts one file. also, if we enable both PyprojectTomlConfigSettingsSource and TomlConfigSettingsSource sources together, it will be confusing.

I would suggest defining pyproject_toml_file for this source.

Would you like to work on it?

[sbrugman]

Thanks for the swift reply. Indeed, pyproject_toml_file is a better naming choice for the configuration.

I'm not able to contribute the feature soon, so if anyone is interested in working on it that would be much appreciated :)