Configuration Management for Python ⚙

Overview

Dynaconf

dynaconf - Configuration Management for Python.

MIT License PyPI PyPI PyPI - Downloads Build Status Azure DevOps builds (branch) Azure DevOps builds (branch) codecov Codacy Badge GitHub issues GitHub stars GitHub Release Date GitHub commits since latest release GitHub last commit Code Style Black Telegram

Foo

Features

  • Inspired by the 12-factor application guide
  • Settings management (default values, validation, parsing, templating)
  • Protection of sensitive information (passwords/tokens)
  • Multiple file formats toml|yaml|json|ini|py and also customizable loaders.
  • Full support for environment variables to override existing settings (dotenv support included).
  • Optional layered system for multi environments [default, development, testing, production]
  • Built-in support for Hashicorp Vault and Redis as settings and secrets storage.
  • Built-in extensions for Django and Flask web frameworks.
  • CLI for common operations such as init, list, write, validate, export.
  • full docs on https://dynaconf.com

Quick start

Install

$ pip install dynaconf

Initialize Dynaconf on project root directory

$ cd path/to/your/project/

$ dynaconf init -f toml

⚙️  Configuring your Dynaconf environment
------------------------------------------
🐍 The file `config.py` was generated.

🎛️  settings.toml created to hold your settings.

🔑 .secrets.toml created to hold your secrets.

🙈 the .secrets.* is also included in `.gitignore`
  beware to not push your secrets to a public repo.

🎉 Dynaconf is configured! read more on https://dynaconf.com

TIP: You can select toml|yaml|json|ini|py on dynaconf init -f <fileformat> toml is the default and also the most recommended format for configuration.

Dynaconf init creates the following files

.
├── config.py       # This is from where you import your settings object (required)
├── .secrets.toml   # This is to hold sensitive data like passwords and tokens (optional)
└── settings.toml   # This is to hold your application setttings (optional)

On the file config.py Dynaconf init generates the following boilerpate

from dynaconf import Dynaconf

settings = Dynaconf(
    envvar_prefix="DYNACONF",  # export envvars with `export DYNACONF_FOO=bar`.
    settings_files=['settings.yaml', '.secrets.yaml'],  # Load files in the given order.
)

TIP: You can create the files yourself instead of using the init command as shown above and you can give any name you want instead of the default config.py (the file must be in your importable python path) - See more options that you can pass to Dynaconf class initializer on https://dynaconf.com

Using Dynaconf

Put your settings on settings.{toml|yaml|ini|json|py}

username = "admin"
port = 5555
database = {name='mydb', schema='main'}

Put sensitive information on .secrets.{toml|yaml|ini|json|py}

password = "secret123"

IMPORTANT: dynaconf init command puts the .secrets.* in your .gitignore to avoid it to be exposed on public repos but it is your responsibility to keep it safe in your local environment, also the recommendation for production environments is to use the built-in support for Hashicorp Vault service for password and tokens.

Optionally you can now use environment variables to override values per execution or per environment.

# override `port` from settings.toml file and automatically casts as `int` value.
export DYNACONF_PORT=9900

On your code import the settings object

from path.to.project.config import settings

# Reading the settings

settings.username == "admin"  # dot notation with multi nesting support
settings.PORT == 9900  # case insensitive
settings['password'] == "secret123"  # dict like access
settings.get("nonexisting", "default value")  # Default values just like a dict
settings.databases.name == "mydb"  # Nested key traversing
settings['databases.schema'] == "main"  # Nested key traversing

More

  • Settings Schema Validation
  • Custom Settings Loaders
  • Vault Services
  • Template substitutions
  • etc...

There is a lot more you can do, read the docs: http://dynaconf.com

Contribute

Main discussions happens on t.me/dynaconf learn more about how to get involved on CONTRIBUTING.md guide

Top Contributors

Issues
  • ARE YOU USING DYNACONF?

    ARE YOU USING DYNACONF?

    Hi,

    As you are on Dynaconf Issues page I assume you are using Dynaconf, or are interested in using it?

    This issue is a placeholder to get feedback from users, can you share your experience?

    Or if you are not a user, can you share what are you looking for in a config management library?

    Questions

    For those using Dynaconf

    • Are you using Dynaconf (yes/no)
    • Can you share some details? (company, project, stack)
    • When did you start using Dynaconf?
    • What features you use?
    • What features you miss?

    For those not using

    • If you have tried Dynaconf but decided not to use, can you share why?
    • If you are still reading and have not tried yet can you share what you are looking for?

    Feel free to share any comment, suggestion you have

    Thanks

    meta 
    opened by rochacbruno 15
  • Community Meeting Meta Issue [BREAKING CHANGES]

    Community Meeting Meta Issue [BREAKING CHANGES]

    Let's discuss the breaking changes coming on Dynaconf 3.0.0 https://doodle.com/poll/ke2g4kc6mzwgtavh

    There are some issues still opened for 3.0 https://github.com/rochacbruno/dynaconf/milestone/9

    The highlights to be discussed are:

    • Global settings object is going to be deprecated Users must create their own instance of dynaconf Deprecate

      from dynaconf import settings
      

      New recommended way

       # yourprogram/config.py
       settings = Dynaconf(**options)`
      

      reason Most users are confused about importing direct from dynaconf a singleton config object and when customizations via **options are needed they have to change the whole codebase.

    • No more default file paths Right now Dynaconf loads all files named settings.* idea is making it explicit only via option settings_files=[list of paths or globs] e.g:

      from dynaconf import Dynaconf
      settings = Dynaconf(settings_files=["main.toml", "other.toml", "path/*.toml"])
      

      reason Debugging file loading is hard, permissions and other I/O problems may happen idea is to make explicit and raise earlier for errors.

    • Envless mode will be default

      Now dynaconf has an envless_mode, which means it can load all variables direct from the main level of a file.

      server = 'bar.com'
      port = 5050
      

      This will be the default and if needed users will provide the list of environments.

      settings = Dynaconf(envs=["development", "production"])
      

      then

      [default]
      server = 'bar.com'
      port = 5050
      [development]
      server = 'dev.com'
      [production]
      server = 'prod.com'
      port = 80
      
    • Allow parser to be changed or disabled Right now toml is the only parser for every variable, idea is to make it configurable (and allow users to opt-out) reason Some users reported that wanted raw values instead of parsed by toml.

    • dotenv will be disabled by default No more loading of .env unless users explicitly enables it. reason In some environments dotenv is already loaded by the shell tool

    • Validators will fire by default if passed to the settings object No need to explicity call settings.validators.validate() if Dynaconf(validators=[...]) is passed it will be called right after the initialization.

    • Remove logging completely reason It is easier to debug using pdb/ipdb, logging right now is useless.

    • Allow Pydantic BaseModels as validators

    Pending Release 
    opened by rochacbruno 14
  • DATABASES setting for Django not being set

    DATABASES setting for Django not being set

    dynaconf is properly showing the settings I put into /etc/pulp/settings.py

    (pulp) [[email protected] pulp]$ dynaconf list -k DATABASES
    Django app detected
    Working in development environment 
    DATABASES: {'default': {'CONN_MAX_AGE': 0,
                 'ENGINE': 'django.db.backends.dummy',
                 'HOST': 'postgres',
                 'NAME': 'pulp',
                 'PORT': '5432',
                 'USER': 'pulp'}}
    

    However, django is not using the setting. It seems like Django is loading the settings for the database before dynaconf is loaded.

    enhancement Pending Release 
    opened by dkliban 13
  • [RFC] Keep @reset mark

    [RFC] Keep @reset mark

    I can still have:

    Dynaconf(merge_enabled=True)

    to have everything merged by default. In fact this my main scenarios, because I do not want to add @merge mark with each setting.

    But in some cases. Especially with lists I'd like default value to be replaced with a new list. Please, keep @reset mark for such cases. Another option is not to merge lists by default or have two options merge_dict_enabled=True and merge_list_enabled=False (this how I would set them).

    Not a Bug RFC 
    opened by dmugtasimov 12
  • What to do with declarations like

    What to do with declarations like "!!python/object/new:box.BoxList" ?

    We use dynaconf to template out a default configuration file in YAML and we've started seeing declarations like these:

    default:
      ALLOWED_HOSTS: !!python/object/new:box.BoxList
        listitems:
        - ::1
        - 127.0.0.1
        - localhost
        state:
          box_class: &id001 !!python/name:box.Box ''
          box_options: {}
          box_org_ref: 140462933942512
    

    I realize these are coming from python-box but they weren't present until recently. They don't look mandatory ? A settings file without these seem to work as well.

    It unfortunately makes the templated file less readable and definitely not human writable.

    enhancement question 
    opened by dmsimard 12
  • Document the use of pytest with dynaconf

    Document the use of pytest with dynaconf

    For testing in my project i want to add in my conftest.py something like that:

    import pytest
    import os
    
    @pytest.fixture(scope='session', autouse=True)
    def settings():
        os.environ['ENV_FOR_DYNACONF'] = 'testing'
    
    

    But this is not work ;-(. What can you advise me ? I dont want start my test like that : ENV_FOR_DYNACONF=testing pytest because somebody can miss that command prefix and mess up some dev data.

    enhancement question Docs good first issue 
    opened by dyens 12
  • Add the ability to load only a specified list of keys from envvars

    Add the ability to load only a specified list of keys from envvars

    Describe the bug Migrating from other configuration systems I'd need to configure envvar_prefix to empty string ("") so I can load any environment variable I used before. But it doesn't seem to work with dynaconf.

    To Reproduce Steps to reproduce the behavior:

    1. Create a basic dynaconf configuration

    In config.py:

    from dynaconf import Dynaconf
    
    settings = Dynaconf(
        # This is the important line.
        envvar_prefix="",
    )
    
    print(settings.FOO)
    
    1. Now create some settings file.

    In settings.py:

    FOO = 'bar'
    
    1. Finally try running.
    export SETTINGS_FILE_FOR_DYNACONF=settings
    FOO='ham' python config.py
    

    You will get:

    bar
    

    Expected behavior

    The print output states:

    ham
    

    This means the value from environment variable has overridden the default value from settings.py module

    Environment (please complete the following information):

    • OS: Ubuntu Linux 20.04
    • Python version: 3.8.5
    • Dynaconf version: 3.1.2
    • Frameworks in use: none.

    Additional context Interestingly running in a "hacky" way changes the variable value:

    _FOO='ham' python config.py
    

    This is not what I'd expect though. The point is to modify the bare FOO variable.

    enhancement help wanted question RFC Docs good first issue HIGH 
    opened by odiroot 11
  • Fix #145 allow string formatting using envvars

    Fix #145 allow string formatting using envvars

    Template substitutions

    Dynaconf has 2 tokens to enable string substitutions @format and @jinja.

    @format token

    Dynaconf allows template substitutions for strings values, by using the @format token prefix and including placeholders accepted by Python's str.format method Dynaconf will call it lazily upon access time.

    The call will be like:

    "<YOURVALUE>".format(env=os.environ, this=dynaconf.settings)
    

    So in your string you can refer to environment variables via env object, and also to variables defined int the settings object itself via this reference. It is lazily evaluated on access it will use the final value for a settings regardless the order of load.

    Example:

    export PROGRAM_NAME=calculator
    

    settings.toml

    [default]
    DB_NAME = "mydb.db"
    
    [development]
    DB_PATH = "@format {env[HOME]}/{this.current_env}/{env[PROGRAM_NAME]}/{this.DB_NAME}"
    
    • {env[HOME]} is the same as os.environ["HOME"] or $HOME in the shell.
    • {this.current_env} is the same as settings.current_env
    • {env[PROGRAM_NAME]} is the same as os.environ["PROGRAM_NAME"] or $PROGRAM_NAME in the shell.
    • {this.DB_NAME} is the same as settins.DB_NAME or settings["DB_NAME"]

    so in your program

    from dynaconf import settings
    
    settings.DB_PATH == '~/development/calculator/mydb.db'
    

    @jinja token

    If jinja2 package is installed then dynaconf will also allow the use jinja to render string values.

    Example:

    export PROGRAM_NAME=calculator
    

    settings.toml

    [default]
    DB_NAME = "mydb.db"
    
    [development]
    DB_PATH = "@jinja {{env.HOME}}/{{this.current_env | lower}}/{{env['PROGRAM_NAME']}}/{{this.DB_NAME}}"
    

    so in your program

    from dynaconf import settings
    
    settings.DB_PATH == '~/development/calculator/mydb.db'
    

    The main difference is that Jinja allows some Python expressions to be avaluated such as {% for, if, while %} and also supports calling methods and has lots of filters like | lower.

    Pending Release 
    opened by rochacbruno 11
  • feat: add filter strategy with PrefixFilter

    feat: add filter strategy with PrefixFilter

    Related to: #621

    This adds a settings_file_prefix option for toml, json, ini and yaml files which works similarly to envvar_prefix, however it defaults to None and does not filter out any keys if it is not set.

    If envs are in use it will do the filtering within each env's variables, an example is shown below

    [default]
    PREFIX_EXTRA = "this will supercede if others are set without prefix"
    
    [development]
    PREFIX_HOST = "https://example.com/development"
    PREFIX_PORT = "8080"
    COMMENT = "this will not show when prefix is used"
    EXTRA = "will not show"
    
    [production]
    PREFIX_HOST = "https://example.com/production"
    PORT = "443"
    EXTRA = "will not show"
    

    When these are loaded up using the settings_file_prefix="PREFIX" option you'll get the following:

    >>> settings.HOST
    'https://example.com/development'
    >>> settings.PORT
    '8080'
    >>> settings.COMMENT
    AttributeError: 'Settings' object has no attribute 'COMMENT'
    

    When switching to production env:

    >>> with settings.using_env("production"):
    ...     settings.HOST
    ...     settings.EXTRA
    ...     settings.PORT
    
    'https://example.com/production'
    'this will supercede if others are set without prefix'
    AttributeError: Attribute PORT was deleted, or belongs to different env
    

    I think this behaves as you'd expect, looking forward to some feedback as I had to fumble about a bit to learn how everything fits together so no doubt I've done some less than ideal things

    Envless load when environments exist in the file will not work in this way as we'd not be able to detect that settings.PRODUCTION is an env to filter the prefix from children. I think this is probably expected if loading envless while envs are included in the file - but if it was possible to check if ENVIRONMENTS_FOR_DYNACONF is False and detect that the top level key is an environment ([production]) then it may be possible, but I could not see an obvious way to detect that production is an environment

    opened by zzZIMAWAKE 10
  • [bug] FlaskDynaconf doesn't handle __contains__

    [bug] FlaskDynaconf doesn't handle __contains__

    Describe the bug

    Checking key in app.config returns False if the key is only set on app.config after the extension is initialized. However, accessing app.config[key] still works. If the key was set before initializing the extension, or through an env var, it correctly reports True.

    This is important for extensions like Flask-SQLAlchemy. It checks if the key SQLALCHEMY_DATABASE_URI is set and shows a warning otherwise, incorrectly in this case.

    To Reproduce

    from dynaconf import FlaskDynaconf
    from flask import Flask
    
    dynaconf = FlaskDynaconf()
    app = Flask(__name__)
    dynaconf.init_app(app)
    app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///app.db"
    
    print("SQLALCHEMY_DATABASE_URI" in app.config)  # False, should be True
    print(app.config["SQLALCHEMY_DATABASE_URI"])  # sqlite:///app.db
    

    Expected behavior

    The in operator reports the correct result when a key is set after calling init_app().

    Environment (please complete the following information):

    • OS: Linux/Arch
    • Dynaconf 3.1.2
    • Flask 1.1.2
    bug 
    opened by davidism 10
  • RFC: Add support for config-files in etc-directories via `dynaconf-xdg plugin`

    RFC: Add support for config-files in etc-directories via `dynaconf-xdg plugin`

    Usually I create a virtualenv for my python-applications, and I collect all config-files in an etc-directory to avoid polluting the project-directory, like this:

    /opt/application-name/bin /opt/application-name/etc/ /opt/application-name/lib /opt/application-name/src/git-checkout

    So I propose to look for config-files not only in the parent-directories, walking all the way up to root, but also check the existence for an etc-subdirectory.

    In bigger projects with lots of components, we split up the etc-dir into .../etc/uwsgi .../etc/supervisord .../etc/dynaconf

    As bonus we also support the global etc-directory: /etc/settings.yml /etc/dynaconf/settings.yml

    enhancement help wanted RFC 
    opened by WouterVH 10
  • [bug]

    [bug]

    When executing an application build by pyinstaller, dynaconf raises OSError: Starting path not found

      File "PyInstaller/loader/pyimod03_importers.py", line 495, in exec_module
      File "myproject/config.py", line 19, in <module>
        settings.validators.register(
      File "dynaconf/base.py", line 113, in __getattr__
      File "dynaconf/base.py", line 163, in _setup
      File "dynaconf/base.py", line 236, in __init__
      File "dynaconf/base.py", line 980, in execute_loaders
      File "dynaconf/loaders/__init__.py", line 160, in settings_loader
      File "dynaconf/base.py", line 1111, in find_file
      File "dynaconf/utils/files.py", line 63, in find_file
      File "dynaconf/utils/files.py", line 13, in _walk_to_root
    

    To Reproduce Steps to reproduce the behavior:

    1. Having the following folder structure
    Project structure
    
    ├── settings.yaml
    ├── myproject
    │   ├── config.py
    
    
    1. Having the following app code:
    Code

    ./myproject/config.py

    settings = Dynaconf(
        envvar_prefix="myproject",
        settings_files=["settings.yaml"],
        environments=True,
        load_dotenv=False,
        env="default",
    )
    settings.validators.register(
    ...
    )
    
    1. Executing under the following environment
    Execution

    Application packaged by pyinstaller (single binary file)

    Environment (please complete the following information):

    • OS: Ubuntu 22.04
    • Dynaconf Version 3.1.8
    bug 
    opened by jurelou 0
  • [Question] Is it possible to get all validation errors in one exception ?

    [Question] Is it possible to get all validation errors in one exception ?

    I want to write a Python script that validate given configs file in my production server pre-deployment.
    My implementation is working, but the raised ValidationError holds only the first encountered error. What I want is to provide feedback in the console about all the keys that did not pass the validation.

    I don't want to use the provided CLI because the script is doing a global check on the projet and I want config validation be a part of it.

    My code looks like this :

        # Validate config
        try:
            from config import config
            # Trigger lazy loading
            config._setup()
        except ValidationError as e:
            click.echo("Error occurred when validating settings files")
            click.echo(str(e))
            sys.exit(1)
    
    

    If this is not possible, can I propose an implementation with a PR ?

    question 
    opened by jidey43 0
  • multiple dotenv

    multiple dotenv

    I love dynaconf. It doesn't support multiple dotenv. I think it should support multiple dotenv.

    for example;

    settings = Dynaconf(
        dotnev_path=[str(BASE_DIR / "envs")],
        settings_files=[".env", ".secret.env"],
    )
    
    question RFC 
    opened by mazlum 0
  • Allow config file to

    Allow config file to "extend" from other files?

    While dynaconf supports including other files by using dynaconf_include variable, it's not good at sloving “extending and overwriting” problems.

    For example, there are 2 config files. The first one is called base.yaml, it contains plenty of common config values for my project:

    APP_CODE: code-a
    APP_SECRET_KEY: some-secret
    OTHER_COMMON_CONFIG: value
    REQUEST_TIMEOUT: 30
    

    Another file is essential_overrides.yaml, it's content is as below:

    REQUEST_TIMEOUT: 1
    

    As you can see: essential_overrides.yaml is not designed to be used on it's own, it overrides a few essential config while most values should inherit from base.yaml.

    To use essential_overrides.yaml as project config, we could try to use the DYNACONF_INCLUDE variable:

    REQUEST_TIMEOUT: 1
    # Add “include” variable
    DYNACONF_INCLUDE:
      - basic.yaml
    

    Sadly, this won't work. Because files defined in DYNACONF_INCLUDE were loaded after regular settings files, So config defined in basic.yaml can never be overwrite by other files. In this case, settings.get('REQUEST_TIMEOUT') will be 30 instead of 1.

    Another workable solution is using multiple settings files. For example, by set “settings files” as below:

    export SETTINGS_FILES_FOR_DYNACONF="base.yaml;essential_overrides.yaml"
    

    The problem can be solved. Although the “extending behavior” was specified in the “multiple settings files”, not some config variable in the file.

    Beside the above solutions, the "layered environments" feature seems like a perfection match for this problem. But in order to use this feature, all config entries must be defined in a single big file. In some situations, it is not doable and it’s better to divide the configs into different files for different environments.

    So, I was wondering that if it’s a good idea to implement a feature called “extending”. It will introduce a new special config attribute called “DYNACONF_EXTEND”. With it’s help, the essential_overrides.yaml can be updated as below:

    REQUEST_TIMEOUT: 1
    
    # "extend" is different from "include". It allows you to override the original config
    # values while “include” will always overwrite config variables in current file.
    DYNACONF_EXTEND:
      - basic.yaml
    
    question 
    opened by piglei 0
  • [bug] Dynaconf loosing VAULT_URL setting from .env file

    [bug] Dynaconf loosing VAULT_URL setting from .env file

    Describe the bug Dynaconf setting object is losing the VAULT_URL and VAULT_VERIFY settings from .env depending on something(unknown) configured in the environment and trying to access the secrets from the localhost:8200 vault server instead.

    To Reproduce Steps to reproduce the behavior:

    1. Having the following folder structure
    Project structure
    # ll -a
    
    -rw-r--r--   1 jitendrayejare  staff    253 May  2 21:25 .codecov.yml
    -rw-r--r--   1 jitendrayejare  staff    195 May  2 21:25 .coveragerc
    -rwxr-xr-x   1 jitendrayejare  staff    340 May  6 00:03 .env
    -rw-r--r--   1 jitendrayejare  staff   1715 May  2 21:25 .env.example
    -rw-r--r--   1 jitendrayejare  staff   1467 May  2 21:25 .gitignore
    -rw-r--r--   1 jitendrayejare  staff    880 May  2 21:25 .pre-commit-config.yaml
    -rw-r--r--   1 jitendrayejare  staff    119 May  2 21:25 .pyup.yml
    -rw-r--r--   1 jitendrayejare  staff    458 May  2 21:25 Dockerfile
    -rw-r--r--   1 jitendrayejare  staff  35121 May  2 21:25 LICENSE
    -rw-r--r--   1 jitendrayejare  staff   7989 May  2 21:25 Makefile
    -rw-r--r--   1 jitendrayejare  staff   1921 May  2 21:25 README.rst
    drwxr-xr-x   3 jitendrayejare  staff     96 May  2 23:41 __pycache__
    -rw-r--r--   1 jitendrayejare  staff   2085 May  2 22:16 broker_settings.yaml
    drwxr-xr-x  84 jitendrayejare  staff   2688 May  6 00:12 conf
    -rw-r--r--   1 jitendrayejare  staff   1977 May  2 21:25 conftest.py
    -rw-r--r--   1 jitendrayejare  staff   1306 May  2 23:42 inventory.yaml
    -rw-r--r--   1 jitendrayejare  staff    554 May  2 21:25 logging.yaml
    -rw-r--r--   1 jitendrayejare  staff    765 May  2 21:25 manage.yml
    -rw-r--r--   1 jitendrayejare  staff    303 May  2 21:25 pyproject.toml
    drwxr-xr-x   4 jitendrayejare  staff    128 May  2 21:25 pytest_fixtures
    drwxr-xr-x  15 jitendrayejare  staff    480 May  2 23:41 pytest_plugins
    -rw-r--r--   1 jitendrayejare  staff    266 May  2 21:25 requirements-optional.txt
    -rw-r--r--   1 jitendrayejare  staff    873 May  2 21:25 requirements.txt
    drwxr-xr-x  29 jitendrayejare  staff    928 May  2 23:41 robottelo
    -rw-r--r--   1 jitendrayejare  staff   1072 May  2 21:25 robottelo.yaml.sample
    drwxr-xr-x  12 jitendrayejare  staff    384 May  2 21:25 scripts
    -rw-r--r--   1 jitendrayejare  staff    416 May  2 21:25 settings.sample.yaml
    -rwxr-xr-x   1 jitendrayejare  staff   1037 May  2 21:25 setup.py
    -rw-r--r--   1 jitendrayejare  staff   3134 May  2 21:25 testimony.yaml
    drwxr-xr-x   7 jitendrayejare  staff    224 May  2 23:48 tests
    
    
    The dynaconf settings object initializing in `robottelo/config/__init__.py` in above file structure. There is also a conf directory that the dynaconf setting object uses to read configutation YAMLs.
    1. Having the following config files:
    Config files

    .env

    VAULT_ENABLED_FOR_DYNACONF=true
    VAULT_URL_FOR_DYNACONF=https://vault.company.com:8200
    VAULT_KV_VERSION_FOR_DYNACONF=2
    VAULT_VERIFY_FOR_DYNACONF=false
    VAULT_MOUNT_POINT_FOR_DYNACONF=apps
    VAULT_PATH_FOR_DYNACONF=dir/subdir
    # VAULT_TOKEN_FOR_DYNACONF=myroot
    # VAULT_ROLE_ID_FOR_DYNACONF=
    # VAULT_SECRET_ID_FOR_DYNACONF=
    

    and

    /conf/azure.yaml

    AZURERM:
      # Client ID of the AzureRM account
      CLIENT_ID: some_client_id
      # Client Secret of the AzureRM account
      CLIENT_SECRET: '@format {this.vault_azure_client_secret}'
    
    1. Having the following app code:
    Code

    /robottelo/config/init.py

    from dynaconf import settings
    
    def get_settings():
        """Return Lazy settings object after validating
    
        :return: A validated Lazy settings object
        """
        settings = LazySettings(
            envvar_prefix="ROBOTTELO",
            core_loaders=["YAML"],
            settings_file="settings.yaml",
            preload=["conf/*.yaml"],
            includes=["settings.local.yaml", ".secrets.yaml", ".secrets_*.yaml"],
            envless_mode=True,
            lowercase_read=True,
            load_dotenv=True,
        )
        settings.validators.register(**VALIDATORS)
    
        try:
            settings.validators.validate()
        except ValidationError as err:
            logger.warning(f'Dynaconf validation failed, continuing for the sake of unit tests\n{err}')
    
        return settings
    
    
    settings = get_settings()
    ...
    
    1. Executing under the following environment
    Execution
    # virtualenv activated
    
    $  pytest tests/foreman/api/test_computeresource_azurerm.py::TestAzureRMComputeResourceTestCase
    
    <eating some big trace here>
    
    ./../../Envs/robotteloPRT/lib/python3.9/site-packages/broker/broker.py:329: in from_inventory
        return [self.reconstruct_host(inv_host) for inv_host in inv_hosts]
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/broker/broker.py:329: in <listcomp>
        return [self.reconstruct_host(inv_host) for inv_host in inv_hosts]
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/broker/broker.py:317: in reconstruct_host
        provider_inst = provider(**host_export_data)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/broker/providers/ansible_tower.py:80: in __init__
        self._validate_settings(instance_name)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/broker/providers/__init__.py:51: in _validate_settings
        settings.execute_loaders()
    ../../DynaConf/dynaconf/dynaconf/base.py:989: in execute_loaders
        core_loader.load(self, env, silent=silent, key=key)
    ../../DynaConf/dynaconf/dynaconf/loaders/vault_loader.py:72: in load
        client = get_client(obj)
    ../../DynaConf/dynaconf/dynaconf/loaders/vault_loader.py:34: in get_client
        client.auth_approle(
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/utils.py:201: in new_func
        return method(*args, **kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/v1/__init__.py:1805: in auth_approle
        return self.login(
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/v1/__init__.py:1495: in login
        return self._adapter.login(url=url, use_token=use_token, **kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/adapters.py:197: in login
        response = self.post(url, **kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/adapters.py:126: in post
        return self.request("post", url, **kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/adapters.py:364: in request
        response = super(JSONAdapter, self).request(*args, **kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/hvac/adapters.py:313: in request
        response = self.session.request(
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/requests/sessions.py:529: in request
        resp = self.send(prep, **send_kwargs)
    ../../../Envs/robotteloPRT/lib/python3.9/site-packages/requests/sessions.py:645: in send
        r = adapter.send(request, **kwargs)
    
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8200): Max retries exceeded with url: /v1/auth/approle/login (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1122120d0>: Failed to establish a new connection: [Errno 61] Connection refused'))
    
    

    Expected behaviour The settings object should read secrets from the given VAULT_URL instead of defaulting to localhost depending upon on some environmental conditions.

    Environment (please complete the following information):

    • OS: [MacOS 12.3.1]
    • Dynaconf Version [3.1.8]
    • Frameworks in use [PyTest]

    Additional context Interestingly this behaviour is only with the execution of tests in a specific environment only(not sure what I am missing or adding there) and not when I attempt to access secrets from the settings object from Ipython(Python Interactive shell) where the secrets are available as attributes of the settings object(settings.vault_azure_client_secret written a secret value from vault).

    bug 
    opened by jyejare 0
  • [bug] Cannot merge using env var with dict that has empty string as its key

    [bug] Cannot merge using env var with dict that has empty string as its key

    Describe the bug I tried to use an env_var to merge a dictionary, but it has the key '' which is not able to work, although I expected it to work.

    To Reproduce I tried to set this variable:

    PULP_LOGGING={'dynaconf_merge': True, 'loggers': {'': {'handlers': ['console'], 'level': 'DEBUG'}}}

    Expected behavior I expected it to merge, but it does not. When showing the effective config with dynaconf list I see it is not merged as expected.

    bug 
    opened by bmbouter 0
Releases(3.1.8)
  • 3.1.3rc1(Mar 1, 2021)

  • 3.1.1(Sep 21, 2020)

    Andreas Poehlmann (1):
          Allow importing SEARCHTREE before settings are configured (#383)
    
    Bruno Rocha (38):
          Hot fix removing unused imports
          Merge branch 'master' of github.com:rochacbruno/dynaconf
          Removing invalid links, adding allert on old docs  fix #369 and fix #371 (#372)
          Fix #359 lazy template substitution on nested keys (#375)
          Flask fizes and other issues included. (#376)
          Fix #379 dict like iteration (#385)
          Fix #377 allow computed values (#386)
          Fix #388 URL reference for custom loaders (#389)
          Fix #382 add is_overriden method (#390)
          Release version 3.1.0
          Create FUNDING.yml
          Fix #391 make box_settings optional, change vendoring strategy (#398)
          HOTFIX: Add missing vendor.txt
          Allow nested Lazy Values (#405)
          Makes   PEP8 more strictly and remove unused variables (#408)
          Merge branch 'master' into vault
          boto is optional
          Merge branch 'vault' into master
          Included example of custom SOPS loader to the docs
          Release version 3.1.1rc1
          HOTFIX: Logging instance has a `formatter` attribute (#410)
          Release version 3.1.1rc2
          Fix set attribute directly and fresh vars (#412)
          384 fix tail and dotted merge (#415)
          Fix #404 no more dup message on combined validators (#417)
          HOTFIX 414 update docs version on release
          Release version 3.1.1rc3
          HOTFIX: Add missing instruction to release.sh
          Added full Dynaconf Diagram and few fizes. (#418)
          Release version 3.1.1rc4
          Small fix on release script
          Minification of vendored modules (#419)
          Release version 3.1.1rc5
          Do not include vendor_src on wheel target (#420)
          Release version 3.1.1rc6
          HOTFIX: Cli now accepts dotter keys
          Release version 3.1.1
    
    Christoph Schmatzler (1):
          Fix typo in Validation docs (#394)
    
    Gabriel Simonetto (1):
          Fix #399 - Update documentation link (#401)
    
    Jiranun Jiratrakanvong (1):
          Add auth username and password for redis settings (#378)
    
    John Vandenberg (1):
          Allow testing against local redis server (#387)
    
    Martijn Pieters (1):
          Correct typos in documentation and README (#400)
    
    Max Winterstein (1):
          Fix typo in release notes (#411)
    
    Mirek Długosz (2):
          Test all names in Validator("foo", "bar", must_exist=False) (#406)
          Fix #407 - add proper equality test for CombinedValidator (#413)
    
    Nikolai Bessonov (1):
          fix a typo (#393)
    
    Peng Yin (5):
          Read all secrets under a vault path
          Add option to auth vault with iam role
          Fix format
          Fix test for versioned kv engine in latest vault
          Merge branch 'master' into vault
    
    Piotr Baniukiewicz (1):
          Fix validation of optional fields (#370)
    
    whg517 (1):
          docs: Fixed filename error in the case of the index page (#396)
    
    Source code(tar.gz)
    Source code(zip)
  • 3.1.1rc6(Sep 21, 2020)

  • 3.1.1rc5(Sep 21, 2020)

  • 3.1.1rc4(Sep 19, 2020)

  • 3.1.1rc3(Sep 17, 2020)

  • 3.1.1rc2(Sep 14, 2020)

  • 3.1.1rc1(Sep 12, 2020)

  • 3.0.0rc2(Jun 22, 2020)

  • 3.0.0rc1(Mar 10, 2020)

  • 2.1.0(Sep 5, 2019)

    Highlights:

    🐲Nested envvars w/ DUNDER__KEYS (useful for #django)

    Lets say you have a configuration like this:

    settings.py

    DATABASES = {
        'default': {
            'NAME': 'db',
            'ENGINE': 'module.foo.engine',
            'ARGS': {'timeout': 30}
        }
    }
    

    And now you want to change the values of ENGINE to other.module, via environment variables you can use the format ${ENVVAR_PREFIX}_${VARIABLE}__${NESTED_ITEM}__${NESTED_ITEM}

    Each __ (dunder, a.k.a double underline) denotes access to nested elements in a dictionary.

    So

    DATABASES['default']['ENGINE'] = 'other.module'
    

    Can be expressed as environment variables as:

    export DYNACONF_DATABASES__default__ENGINE=other.module
    

    NOTE: if you are using Django extension then the prefix will be DJANGO_ instead of DYNACONF_ and the same if you are using FLASK_ or a custom prefix if you have customized the ENVVAR_PREFIX.

    This will result in

    DATABASES = {
        'default': {
            'NAME': 'db',
            'ENGINE': 'other.module',
            'ARGS': {'timeout': 30}
        }
    }
    

    Read more on: https://dynaconf.readthedocs.io/en/latest/guides/environment_variables.html#nested-keys-in-dictionaries-via-environment-variables

    🔃.from_env easy access to different envs

    Return a new isolated settings object pointing to specified env.

    Example of settings.toml::

    [development]
    message = 'This is in dev'
    foo = 1
    [other]
    message = 'this is in other env'
    bar = 2
    

    Then you can use from_env:

    >>> print(settings.from_env('other').MESSAGE)
    'This is in other env'
    >>> print(settings.from_env('other').BAR)
    2
    >>> print(settings.from_env('other').FOO)
    AttributeError: settings object has no attribute 'FOO'
    

    The existing settings object remains the same.

    >>> print(settings.MESSAGE)
    'This is in dev'
    

    Read more on: https://dynaconf.readthedocs.io/en/latest/guides/advanced_usage.html#from-env

    📋$dynaconf list -o export your settings as a file

    dynaconf list -o path/to/file.yaml
    

    The above command will export all the items showed by dynaconf list to the desired format which is inferred by the -o file extension, supported formats yaml, toml, ini, json, py

    When using py you may want a flat output (without being nested inside the env key)

    dynaconf list -o path/to/file.py --output-flat
    

    Read more on: https://dynaconf.readthedocs.io/en/latest/guides/cli.html#exporting-current-environment-as-a-file

    🔐@HashiCorp #Vault & @RedisLabs supports multiple envs

    If you want to write to specific env pass the -e option.

    $ dynaconf write redis -v name=Bruno -v database=localhost -v port=1234 -e production
    

    The above data will be recorded in redis as a hash:

    DYNACONF_PRODUCTION {
        NAME='Bruno'
        DATABASE='localhost'
        PORT='@int 1234'
    }
    

    Then to access that values you can set export ENV_FOR_DYNACONF=production or directly via settings.from_env('production').NAME

    Read more on: https://dynaconf.readthedocs.io/en/latest/guides/external_storages.html

    Dynaconf 2.1.0

    Bruno Rocha (8):
          Release version 2.0.4
          Merge branch 'dgarcia360-master'
          Fix #197 add support for DOTTED__ENV__VARS (#215)
          Add support to export merged env to filesystem via cli. (#217)
          Adds `from_env` method and change `_store` to be a `DynaBox` (#219)
          hotfix: next release will be 2.1.0 because new features added. (#220)
          Fix `make test_examples` to use better assertions, redis and vault loader now respects `envs` (#222)
          fix #221 removed JSON,YAML,INI,TOML cosntants from default_settings (#223)
    
    Kedar Kulkarni (1):
          Add `list_envs` function to vault loader and now envs can have `_` on its name.
    
    Pavel Alimpiev (1):
          Fix typo in documentation for a Validator class (#213)
    
    dgarcia360 (3):
          Updated configuration options table to csv table
          Added responsive table fix
          Fix format
    
    Source code(tar.gz)
    Source code(zip)
  • 2.0.0(Apr 9, 2019)

    Dynaconf 2.0.0

    • Fix #129 get_fresh should be case insensitive
    • Fix #125 .configure was not loading settings_module passed as argument
    • Fix #127 fix YAML warnings and default to full_load
    • Allow disable of core loaders #122
    • Added support for Jenkins secrets file #117
    • Added more examples for includes #110
    • Moved to Azure Pipelines CI #142
    • Added 100% test coverage on windows (Unit & Functional tests)
    • Deprecated MERGE_ENABLED in favor of local dynaconf_merge
    • Fix #74 - Better File Searching (now building a reasonable Search Tree)
    • Now it finds settings when invoking from out of Script folder
    • Fixed test environment (each test now run in a separate tmpdir)
    • Added a check to avoid Circular references when starting settings inside settings
    • Added Django Extension v2 with better syntax and a lot od inspect instrospetion
    • Updated documentation about new features
    • Added a not that YAML is the recommended format for Django
    • Added support for Django Standalone Script
    • Added support for Django unit testing
    • Fix #148 env was not being passed to custom loaders
    • Fix #144 removed six as it is a Py3.4+ only project
    • Added Backwards compatibility for users using old django Extension
    • start_dotenv is now Lazy (only when settings._setup is called)
    • Added new _FOR_DYNACONF config options ENV_SWITCHER, SKIP_FILES, INCLUDES, CORE_LOADERS, SECRETS
    • Renamed config PROJECT_ROOT -> ROOT_PATH
    Source code(tar.gz)
    Source code(zip)
Owner
Bruno Rocha
Programmer at @RedHatOfficial. #Python #Rust . Working on: @ansible @python #dynaconf @codeshow
Bruno Rocha
Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support.

Simple dataclasses configuration management for Python with hocon/json/yaml/properties/env-vars/dict support, based on awesome and lightweight pyhocon parsing library.

Teo Stocco 43 May 11, 2022
Pydantic-ish YAML configuration management.

Pydantic-ish YAML configuration management.

Dribia Data Research 16 Apr 13, 2022
filetailor is a peer-based configuration management utility for plain-text files such as dotfiles.

filetailor filetailor is a peer-based configuration management utility for plain-text files (and directories) such as dotfiles. Files are backed up to

null 4 Jan 15, 2022
Flexible Python configuration system. The last one you will ever need.

OmegaConf Description Project Code quality Docs and support OmegaConf is a hierarchical configuration system, with support for merging configurations

Omry Yadan 1.2k May 25, 2022
Python Marlin Configurator to make valid configuration files to be used to compile Marlin with.

marlin-configurator Concept originally imagined by The-EG using PowerShell Build Script for Marlin Configurations The purpose of this project is to pa

DevPeeps 2 Oct 9, 2021
A Python library to parse PARI/GP configuration and header files

pari-utils A Python library to parse PARI/GP configuration and header files. This is mainly used in the code generation of https://github.com/sagemath

Sage Mathematical Software System 2 Dec 30, 2021
Configuration Extractor for EXE4J PE files

EXE4J Configuration Extractor This script helps reverse engineering Portable Executable files created with EXE4J by extracting their configuration dat

Karsten Hahn 5 Feb 11, 2022
Sync any your configuration file to remote. Currently only support gist.

Sync your configuration to remote, such as vimrc. You can use EscSync to manage your configure of editor, shell, etc.

Me1onRind 1 Nov 28, 2021
Tools to assist with the configuration and maintenance of fapolicyd.

Tools to assist with the configuration and maintenance of fapolicyd.

Concurrent Technologies Corporation (CTC) 5 Jan 17, 2022
A tool to manage configuration files, build scripts etc. across multiple projects.

A tool to manage configuration files, build scripts etc. across multiple projects.

null 6 Feb 28, 2022
An application pulls configuration information from JSON files generated

AP Provisioning Automation An application pulls configuration information from JSON files generated by Ekahau and then uses Netmiko to configure the l

Cisco GVE DevNet Team 1 Dec 17, 2021
KConfig Browser is a graphical application which allows you to modify KDE configuration files found in ~/.config

kconfig_browser KConfig Browser is a graphical application which allows you to modify KDE configuration files found in ~/.config Screenshot Why I crea

null 10 Dec 29, 2021
Secsie is a configuration language made for speed, beauty, and ease of use.

secsie-conf pip3 install secsie-conf Secsie is a configuration language parser for Python, made for speed and beauty. Instead of writing config files

Noah Broyles 3 Feb 19, 2022
A slightly opinionated template for iPython configuration for interactive development

A slightly opinionated template for iPython configuration for interactive development. Auto-reload and no imports for packages and modules in the project.

Seva Zhidkov 24 Feb 16, 2022
Python 3+ compatible port of the configobj library

configobj Python 3+ compatible port of the configobj library. Documentation You can find a full manual on how to use ConfigObj at readthedocs. If you

Differently Sized Kittens 273 May 29, 2022
🤫 Easily manage configs and secrets in your Python projects (with CLI support)

Installation pip install confidential How does it work? Confidential manages secrets for your project, using AWS Secrets Manager. First, store a secr

Candid™️ 61 May 22, 2022
A modern simfile parsing & editing library for Python 3

A modern simfile parsing & editing library for Python 3

ash garcia 30 May 11, 2022
Scooch Configures Object Oriented Class Hierarchies for python

Scooch Scooch Configures Object Oriented Class Hierarchies for python. A good place to start with Scooch is at the documentation found here. Scooch is

Pandora Media, Inc. 5 Mar 12, 2022
A set of Python scripts and notebooks to help administer and configure Workforce projects.

Workforce Scripts A set of Python scripts and notebooks to help administer and configure Workforce projects. Notebooks Several example Jupyter noteboo

Esri 73 May 20, 2022