ConfZ – Pydantic Config Management
ConfZ is a configuration management library for Python based on pydantic. It easily allows you to
- load your configuration from config files, environment variables, command line arguments and more sources
- transform the loaded data into a desired format and validate it
- access the results as Python dataclass-like objects with full IDE support
It furthermore supports you in common use cases like:
- Multiple environments
- Singleton with lazy loading
- Config changes for unit tests
- Custom config sources
📦
Installation
ConfZ is on PyPI and can be installed with pip:
pip install confz
🚀
Quick Start
The first step of using ConfZ is to declare your config classes and sources, for example in config.py:
from pathlib import Path
from confz import ConfZ, ConfZFileSource
from pydantic import SecretStr, AnyUrl
class DBConfig(ConfZ):
user: str
password: SecretStr
class APIConfig(ConfZ):
host: AnyUrl
port: int
db: DBConfig
CONFIG_SOURCES = ConfZFileSource(file=Path('/path/to/config.yml'))
Thanks to pydantic, you can use a wide variety of field types and validators.
From now on, in any other file, you can access your config directly:
from config import APIConfig
print(f"Serving API at {APIConfig().host}, port {APIConfig().port}.")
As can be seen, the config does neither have to be loaded explicitly, nor instantiated globally. ConfZ automatically loads your config as defined in CONFIG_SOURCES the first time you access it. Thanks to its singleton mechanism, this happens the first time only, afterwards you get back a cached, immutable instance, behaving like any other pydantic instance.
assert APIConfig() is APIConfig() # true because of singleton mechanism
APIConfig().port = 1234 # raises an error because of immutability
APIConfig().json() # call pydantic's method to get a json representation
Note: While the implicit and hidden loading of your config might be surprising and feel a bit like Python magic at first, it allows you to reduce a lot of boilerplate. Instead of having to load your config explicitly and then passing it down to all code layers that need it, you can directly access it from anywhere by just importing your config class and accessing for example APIConfig().db.user directly.
More Config Sources
ConfZ is highly flexible in defining the source of your config. Do you have multiple environments? No Problem:
from pathlib import Path
from confz import ConfZ, ConfZFileSource
class MyConfig(ConfZ):
...
CONFIG_SOURCES = ConfZFileSource(
folder=Path('/path/to/config/folder'),
file_from_env='ENVIRONMENT'
)
Your config file can now be defined in the environment variable ENVIRONMENT and is relative to folder.
You can also provide a list as config source and read for example from environment variables and from command line arguments:
from confz import ConfZ, ConfZEnvSource, ConfZCLArgSource
class MyConfig(ConfZ):
...
CONFIG_SOURCES = [
ConfZEnvSource(allow_all=True),
ConfZCLArgSource(prefix='conf_')
]
ConfZ now tries to populate your config either from environment variables having the same name as your attributes or by reading command line arguments that start with conf_. Recursive models are supported too, for example if you want to control the user-name in the API above, you can either set the environment variable DB.USER or pass the command line argument --conf_db.user.
Explicit Loading
In some scenarios, the config should not be a global singleton, but loaded explicitly and passed around locally. Instead of defining CONFIG_SOURCES as class variable, the sources can also be defined in the constructor directly:
from pathlib import Path
from confz import ConfZ, ConfZFileSource, ConfZEnvSource
class MyConfig(ConfZ):
number: int
text: str
config1 = MyConfig(config_sources=ConfZFileSource(file=Path('/path/to/config.yml')))
config2 = MyConfig(config_sources=ConfZEnvSource(prefix='CONF_', allow=['text']), number=1)
config3 = MyConfig(number=1, text='hello world')
As can be seen, additional keyword-arguments can be provided as well.
Note: If neither class variable CONFIG_SOURCES nor constructor argument config_sources is provided, ConfZ behaves like a regular pydantic class.
Change Config Values
In some scenarios, you might want to change your config values, for example within a unit test. However, if you set the CONFIG_SOURCES class variable, this is not directly possible. To overcome this, every config class provides a context manager to temporarily change your config:
from pathlib import Path
from confz import ConfZ, ConfZFileSource, ConfZDataSource
class MyConfig(ConfZ):
number: int
CONFIG_SOURCES = ConfZFileSource(file=Path('/path/to/config.yml'))
print(MyConfig().number) # will print the value from the config-file
new_source = ConfZDataSource(data={'number': 42})
with MyConfig.change_config_sources(new_source):
print(MyConfig().number) # will print '42'
print(MyConfig().number) # will print the value from the config-file again
Early Validation
By default, your config gets loaded the first time you instantiate the class, e.g. with MyConfig().attribute. This prevents side effects like loading a file while you import your config classes. If the config class cannot populate all mandatory fields in the correct format, pydantic will raise an error at this point. To make sure this does not happen in an inconvenient moment, you can also instruct ConfZ to load all configs beforehand:
from confz import validate_all_configs
if __name__ == '__main__':
validate_all_configs()
# your application code
The function validate_all_configs will instantiate all config classes defined in your code at any (reachable) location that have CONFIG_SOURCES set.
📖
Documentation
Now you've seen the two ways how ConfZ can be used: With class variable config sources, unlocking a singleton with lazy loading, or with keyword argument config sources, allowing to directly load your config values. In both cases, defining your config sources from files, command line arguments and environment variables is highly flexible (and also extendable, by the way), while pydantic still makes sure that everything matches your expectations in the end. You've also seen how to temporarily change your config for example in unit tests and how to validate your singleton config classes early in the code already.
The full documentation of ConfZ's features can be found at readthedocs.
ℹ️
About
ConfZ was programmed and will be maintained by Zühlke. The first version was realized by Silvan. Special thanks to Iwan with his ConfMe, which inspired this project.
Want to contribute to ConfZ? Check out the contribution instruction & guidelines.
![Toml file badly processed if it contains table aka. section in .ini files (name in brackets [])](https://avatars.githubusercontent.com/u/19777909?v=4)
5 Dec 23, 2022
2.8k Jan 6, 2023
62 Dec 23, 2022
3 Sep 18, 2022
1.4k Jan 2, 2023
2 Oct 9, 2021
6 Jun 29, 2022
0 Nov 21, 2022
7 Dec 27, 2022
8 Dec 14, 2022
1 Dec 17, 2021
11 Sep 15, 2022
3 Feb 19, 2022
24 Feb 16, 2022
288 Dec 14, 2022
38 Nov 1, 2022
920 Jan 4, 2023
2 Jan 8, 2022
2 Jan 20, 2022