A drop-in replacement for argparse that allows options to also be set via config files and/or environment variables.

Related tags

ConfigArgParse
Overview

ConfigArgParse

PyPI version Supported Python versions Travis CI build

Overview

Applications with more than a handful of user-settable options are best configured through a combination of command line args, config files, hard-coded defaults, and in some cases, environment variables.

Python's command line parsing modules such as argparse have very limited support for config files and environment variables, so this module extends argparse to add these features.

Available on PyPI: http://pypi.python.org/pypi/ConfigArgParse

https://travis-ci.org/bw2/ConfigArgParse.svg?branch=master

Features

  • command-line, config file, env var, and default settings can now be defined, documented, and parsed in one go using a single API (if a value is specified in more than one way then: command line > environment variables > config file values > defaults)
  • config files can have .ini or .yaml style syntax (eg. key=value or key: value)
  • user can provide a config file via a normal-looking command line arg (eg. -c path/to/config.txt) rather than the argparse-style @config.txt
  • one or more default config file paths can be specified (eg. ['/etc/bla.conf', '~/.my_config'] )
  • all argparse functionality is fully supported, so this module can serve as a drop-in replacement (verified by argparse unittests).
  • env vars and config file keys & syntax are automatically documented in the -h help message
  • new method print_values() can report keys & values and where they were set (eg. command line, env var, config file, or default).
  • lite-weight (no 3rd-party library dependencies except (optionally) PyYAML)
  • extensible (ConfigFileParser can be subclassed to define a new config file format)
  • unittested by running the unittests that came with argparse but on configargparse, and using tox to test with Python 2.7 and Python 3+

Example

my_script.py:

Script that defines 4 options and a positional arg and then parses and prints the values. Also, it prints out the help message as well as the string produced by format_values() to show what they look like.

import configargparse

p = configargparse.ArgParser(default_config_files=['/etc/app/conf.d/*.conf', '~/.my_settings'])
p.add('-c', '--my-config', required=True, is_config_file=True, help='config file path')
p.add('--genome', required=True, help='path to genome file')  # this option can be set in a config file because it starts with '--'
p.add('-v', help='verbose', action='store_true')
p.add('-d', '--dbsnp', help='known variants .vcf', env_var='DBSNP_PATH')  # this option can be set in a config file because it starts with '--'
p.add('vcf', nargs='+', help='variant file(s)')

options = p.parse_args()

print(options)
print("----------")
print(p.format_help())
print("----------")
print(p.format_values())    # useful for logging where different settings came from

config.txt:

Since the script above set the config file as required=True, lets create a config file to give it:

# settings for my_script.py
genome = HCMV     # cytomegalovirus genome
dbsnp = /data/dbsnp/variants.vcf

command line:

Now run the script and pass it the config file:

DBSNP_PATH=/data/dbsnp/variants_v2.vcf python config_test.py --my-config config.txt f1.vcf f2.vcf

output:

Here is the result:

Namespace(dbsnp='/data/dbsnp/variants_v2.vcf', genome='HCMV', my_config='config.txt', v=False, vcf=['f1.vcf', 'f2.vcf'])
----------
usage: config_test.py [-h] -c MY_CONFIG --genome GENOME [-v] [-d DBSNP]
                      vcf [vcf ...]

Args that start with '--' (eg. --genome) can also be set in a config file
(/etc/app/conf.d/*.conf or ~/.my_settings or specified via -c). Config file
syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at
https://goo.gl/R74nmi). If an arg is specified in more than one place, then
commandline values override environment variables which override config file
values which override defaults.

positional arguments:
  vcf                   variant file(s)

optional arguments:
  -h, --help            show this help message and exit
  -c MY_CONFIG, --my-config MY_CONFIG
                        config file path
  --genome GENOME       path to genome file
  -v                    verbose
  -d DBSNP, --dbsnp DBSNP
                        known variants .vcf [env var: DBSNP_PATH]

----------
Command Line Args:   --my-config config.txt f1.vcf f2.vcf
Environment Variables:
  DBSNP_PATH:        /data/dbsnp/variants_v2.vcf
Config File (config.txt):
  genome:            HCMV

Special Values

Under the hood, configargparse handles environment variables and config file values by converting them to their corresponding command line arg. For example, "key = value" will be processed as if "--key value" was specified on the command line.

Also, the following special values (whether in a config file or an environment variable) are handled in a special way to support booleans and lists:

  • key = true is handled as if "--key" was specified on the command line. In your python code this key must be defined as a boolean flag (eg. action="store_true" or similar).
  • key = [value1, value2, ...] is handled as if "--key value1 --key value2" etc. was specified on the command line. In your python code this key must be defined as a list (eg. action="append").

Config File Syntax

Only command line args that have a long version (eg. one that starts with '--') can be set in a config file. For example, "--color" can be set by putting "color=green" in a config file. The config file syntax depends on the constuctor arg: config_file_parser_class which can be set to one of the provided classes: DefaultConfigFileParser, YAMLConfigFileParser, ConfigparserConfigFileParser or to your own subclass of the ConfigFileParser abstract class.

DefaultConfigFileParser - the full range of valid syntax is:

# this is a comment
; this is also a comment (.ini style)
---            # lines that start with --- are ignored (yaml style)
-------------------
[section]      # .ini-style section names are treated as comments

# how to specify a key-value pair (all of these are equivalent):
name value     # key is case sensitive: "Name" isn't "name"
name = value   # (.ini style)  (white space is ignored, so name = value same as name=value)
name: value    # (yaml style)
--name value   # (argparse style)

# how to set a flag arg (eg. arg which has action="store_true")
--name
name
name = True    # "True" and "true" are the same

# how to specify a list arg (eg. arg which has action="append")
fruit = [apple, orange, lemon]
indexes = [1, 12, 35 , 40]

YAMLConfigFileParser - allows a subset of YAML syntax (http://goo.gl/VgT2DU)

# a comment
name1: value
name2: true    # "True" and "true" are the same

fruit: [apple, orange, lemon]
indexes: [1, 12, 35, 40]

ConfigparserConfigFileParser - allows a subset of python's configparser module syntax (https://docs.python.org/3.7/library/configparser.html). In particular the following configparser options are set:

config = configparser.ConfigParser(
    delimiters=("=",":"),
    allow_no_value=False,
    comment_prefixes=("#",";"),
    inline_comment_prefixes=("#",";"),
    strict=True,
    empty_lines_in_values=False,
)

Once configparser parses the config file all section names are removed, thus all keys must have unique names regardless of which INI section they are defined under. Also, any keys which have python list syntax are converted to lists by evaluating them as python code using ast.literal_eval (https://docs.python.org/3/library/ast.html#ast.literal_eval). To facilitate this all multi-line values are converted to single-line values. Thus multi-line string values will have all new-lines converted to spaces. Note, since key-value pairs that have python dictionary syntax are saved as single-line strings, even if formatted across multiple lines in the config file, dictionaries can be read in and converted to valid python dictionaries with PyYAML's safe_load. Example given below:

# inside your config file (e.g. config.ini)
[section1]  # INI sections treated as comments
system1_settings: { # start of multi-line dictionary
    'a':True,
    'b':[2, 4, 8, 16],
    'c':{'start':0, 'stop':1000},
    'd':'experiment 32 testing simulation with parameter a on'
    } # end of multi-line dictionary value

.......

# in your configargparse setup
import configargparse
import yaml

parser = configargparse.ConfigParser(
    config_file_parser_class=configargparse.ConfigparserConfigFileParser
)
parser.add_argument('--system1_settings', type=yaml.safe_load)

args = parser.parse_args() # now args.system1 is a valid python dict

ArgParser Singletons

To make it easier to configure different modules in an application, configargparse provides globally-available ArgumentParser instances via configargparse.get_argument_parser('name') (similar to logging.getLogger('name')).

Here is an example of an application with a utils module that also defines and retrieves its own command-line args.

main.py

import configargparse
import utils

p = configargparse.get_argument_parser()
p.add_argument("-x", help="Main module setting")
p.add_argument("--m-setting", help="Main module setting")
options = p.parse_known_args()   # using p.parse_args() here may raise errors.

utils.py

import configargparse
p = configargparse.get_argument_parser()
p.add_argument("--utils-setting", help="Config-file-settable option for utils")

if __name__ == "__main__":
   options = p.parse_known_args()

Help Formatters

ArgumentDefaultsRawHelpFormatter is a new HelpFormatter that both adds default values AND disables line-wrapping. It can be passed to the constructor: ArgParser(.., formatter_class=ArgumentDefaultsRawHelpFormatter)

Aliases

The configargparse.ArgumentParser API inherits its class and method names from argparse and also provides the following shorter names for convenience:

  • p = configargparse.get_arg_parser() # get global singleton instance
  • p = configargparse.get_parser()
  • p = configargparse.ArgParser() # create a new instance
  • p = configargparse.Parser()
  • p.add_arg(..)
  • p.add(..)
  • options = p.parse(..)

HelpFormatters:

  • RawFormatter = RawDescriptionHelpFormatter
  • DefaultsFormatter = ArgumentDefaultsHelpFormatter
  • DefaultsRawFormatter = ArgumentDefaultsRawHelpFormatter

Design Notes

Unit tests:

tests/test_configargparse.py contains custom unittests for features specific to this module (such as config file and env-var support), as well as a hook to load and run argparse unittests (see the built-in test.test_argparse module) but on configargparse in place of argparse. This ensures that configargparse will work as a drop in replacement for argparse in all usecases.

Previously existing modules (PyPI search keywords: config argparse):

  • argparse (built-in module Python v2.7+)
    • Good:
      • fully featured command line parsing
      • can read args from files using an easy to understand mechanism
    • Bad:
      • syntax for specifying config file path is unusual (eg. @file.txt)and not described in the user help message.
      • default config file syntax doesn't support comments and is unintuitive (eg. --namevalue)
      • no support for environment variables
  • ConfArgParse v1.0.15 (https://pypi.python.org/pypi/ConfArgParse)
    • Good:
      • extends argparse with support for config files parsed by ConfigParser
      • clear documentation in README
    • Bad:
      • config file values are processed using ArgumentParser.set_defaults(..) which means "required" and "choices" are not handled as expected. For example, if you specify a required value in a config file, you still have to specify it again on the command line.
      • doesn't work with Python 3 yet
      • no unit tests, code not well documented
  • appsettings v0.5 (https://pypi.python.org/pypi/appsettings)
    • Good:
      • supports config file (yaml format) and env_var parsing
      • supports config-file-only setting for specifying lists and dicts
    • Bad:
      • passes in config file and env settings via parse_args namespace param
      • tests not finished and don't work with Python 3 (import StringIO)
  • argparse_config v0.5.1 (https://pypi.python.org/pypi/argparse_config)
    • Good:
      • similar features to ConfArgParse v1.0.15
    • Bad:
      • doesn't work with Python 3 (error during pip install)
  • yconf v0.3.2 - (https://pypi.python.org/pypi/yconf) - features and interface not that great
  • hieropt v0.3 - (https://pypi.python.org/pypi/hieropt) - doesn't appear to be maintained, couldn't find documentation
  • configurati v0.2.3 - (https://pypi.python.org/pypi/configurati)
    • Good:
      • JSON, YAML, or Python configuration files
      • handles rich data structures such as dictionaries
      • can group configuration names into sections (like .ini files)
    • Bad:
      • doesn't work with Python 3
      • 2+ years since last release to PyPI
      • apparently unmaintained

Design choices:

  1. all options must be settable via command line. Having options that can only be set using config files or env. vars adds complexity to the API, and is not a useful enough feature since the developer can split up options into sections and call a section "config file keys", with command line args that are just "--" plus the config key.
  2. config file and env. var settings should be processed by appending them to the command line (another benefit of #1). This is an easy-to-implement solution and implicitly takes care of checking that all "required" args are provied, etc., plus the behavior should be easy for users to understand.
  3. configargparse shouldn't override argparse's convert_arg_line_to_args method so that all argparse unit tests can be run on configargparse.
  4. in terms of what to allow for config file keys, the "dest" value of an option can't serve as a valid config key because many options can have the same dest. Instead, since multiple options can't use the same long arg (eg. "--long-arg-x"), let the config key be either "--long-arg-x" or "long-arg-x". This means the developer can allow only a subset of the command-line args to be specified via config file (eg. short args like -x would be excluded). Also, that way config keys are automatically documented whenever the command line args are documented in the help message.
  5. don't force users to put config file settings in the right .ini [sections]. This doesn't have a clear benefit since all options are command-line settable, and so have a globally unique key anyway. Enforcing sections just makes things harder for the user and adds complexity to the implementation.
  6. if necessary, config-file-only args can be added later by implementing a separate add method and using the namespace arg as in appsettings_v0.5

Relevant sites:

Versioning

This software follows Semantic Versioning

Issues
  • Allow some command line options to be merged with those in the config

    Allow some command line options to be merged with those in the config

    Taking from the first point of Features section of the readme:

    If a value is specified in more than one way then: command line > environment variables > config file values > defaults.

    ConfigArgParse (CAP) overrides the values specified in the config file with those specified in the command line arguments. For most of the cases this behaviour is nice, except when it isn't.

    Consider the case of a mitmproxy user who has specified some scripts in the config file, and when they specify another script in the command line, the config ones get replaced.

    The expected behaviour here is the merging of the list of scripts.

    I propose that CAP should add a new parameter (say multiple) to parser.add_argument that determines this override behaviour. So, for eg, the following snippet would result in an option that merges the values:

    parser.add_argument(
        "-s", "--script", 
        action="append", type=str, dest="scripts" default=[],
        multiple="merge"
    )
    

    The multiple parameter could have values:

    • merge - Merge the values specified in the command line, env variables & config files & defaults.
    • override - Follow the normal override process we follow now.
    opened by dufferzafar 14
  • bug when setting a float value to -10. and not -10.0

    bug when setting a float value to -10. and not -10.0

    When fixing a value defined with: parser.add('--'+par_name,type=types.FloatType,required=False,help=par_help, default=par_default)

    I get an error if in the config file i set par_name = -10. or par_name = -9. The error doesn't occur in the following: ` par_name = -10.0

    par_name = 10.

    par_name = 10.0 `

    opened by londumas 13
  • Refactor config file parsing

    Refactor config file parsing

    This is a basic refactoring to address #21, I'm working on testing it with custom parser now... All tests are passing, so hopefully I did not mess up anything.

    The documentation and examples in the README will be updated based on input from others.

    opened by lahwaacz 11
  • Support for Python2.6 (without argparse tests)

    Support for Python2.6 (without argparse tests)

    partially fixes #17

    opened by kuba 10
  • Feature/support argcomplete

    Feature/support argcomplete

    Add support for argcomplete (https://github.com/kislyuk/argcomplete) on ConfigArgParse See kislyuk/argcomplete#123 for specific usecases.

    I think this pr will solve the problem with action="append". The problem was that argcomplete create action new class that inherent from the old one:

    class IntrospectAction(action.__class__):
       ...
    

    This change is Reviewable

    opened by roi-meir 9
  • Update Python versions

    Update Python versions

    Python 2.6, 3.2 and 3.3 are EOL and no longer receiving security updates.

    They're also little used if at all.

    Here's the pip installs for ConfigArgParse from PyPI for last month:

    | python_version | percent | download_count | | -------------- | ------: | -------------: | | 2.7 | 88.83% | 150,131 | | 3.4 | 5.78% | 9,762 | | 3.6 | 3.08% | 5,203 | | 3.5 | 2.00% | 3,379 | | 2.6 | 0.25% | 417 | | 3.7 | 0.04% | 63 | | None | 0.03% | 44 | | 3.3 | 0.01% | 18 |

    Source: pypinfo --start-date -46 --end-date -19 --percent --pip --markdown ConfigArgParse pyversion

    This removes those, and updates the code to use more modern Python features. It also adds Python 3.6.

    opened by hugovk 9
  • Dead repo? Still maintained?

    Dead repo? Still maintained?

    Hi, No commits have been made since quite long now. No pull requests accepted. @bw2 - Do you still plan to maintain this project? Do you need any help? I can assist if needed. Cheers, Ronan

    opened by ronhanson 9
  • Use pytest to run all tests

    Use pytest to run all tests

    ConfigArgParse uses setuptools (using python setup.py test) to run all tests. However, this gives a WARNING: Testing via this command is deprecated and will be removed in a future version.. pytest is said to be an drop-in replacement for this.

    Running pytest reveals a few minor issues (1665 of the 1666 tests work fine). I'll enumerate them here.

    testConfigOrEnvValueErrors fails

    (tests.test_configargparse.TestBasicUseCases.testConfigOrEnvValueErrors)

    testConfigOrEnvValueErrors SUCCEEDS for python setup.py test testConfigOrEnvValueErrors SUCCEEDS for pytest or pytest -k testConfigOrEnvValueErrors testConfigOrEnvValueErrors FAILS for pytest -v or pytest -v -k testConfigOrEnvValueErrors

    The -v parameter means that pytest is verbose. testConfigOrEnvValueErrors tests the VERBOSE environment variable (amongst other things). I suspect that this test incorrectly fails due to the way pytest sets or influences by the environment variables.

    TestHelpFormattingMetaclass gives a warning

    (tests.test_argparse.TestHelpFormattingMetaclass)

    Pytests gives a PytestCollectionWarning: cannot collect test class 'TestHelpFormattingMetaclass' because it has a __init__ constructor (from: tests/test_configargparse.py).

    This is a bit of a pickle. TestHelpFormattingMetaclass is defined in Python's test_argparse.py, and we can't change the code.

    This class contains a few tests that we perhaps like to run, but neither unittest nor pytest collects any of these tests because these tests are dynamically added to the class, and seemingly neither unittest nor pytest can handle that situation. The only difference is that pytest gives a warning about it.

    I tried to suppress the warning by setting

    test.test_argparse.TestHelpFormattingMetaclass.__test__ = False
    

    But so far, I still see the warning.

    testGlobalInstances gives confusing output

    (tests.test_configargparse.TestMisc.testGlobalInstances and tests.test_configargparse.TestMisc.testGlobalInstances_WithName)

    These tests both contain code that tests if a ValueError is raised under certain conditions (in the self.assertRaisesRegex line). This seems to work fine, and the tests seem to pass. At least, they always pass according to the summary line and the lines

    tests/test_configargparse.py::TestMisc::testGlobalInstances PASSED
    tests/test_configargparse.py::TestMisc::testGlobalInstances_WithName PASSED
    

    If only theses two tests are run with pytest -k testGlobalInstances -v, there are no errors, even with the verbose (-v) option set. However, if all tests are run with pytest -v, it presents us with two errors in the stdout:

    testGlobalInstances (tests.test_configargparse.TestMisc) ... ERROR
    testGlobalInstances_WithName (tests.test_configargparse.TestMisc) ... ERROR
    

    In addition, the two ValueErrror exceptions are shows in the test summary, even though the tests passes:

    tests/test_configargparse.py::TestMisc::testGlobalInstances PASSED
    tests/test_configargparse.py::TestMisc::testGlobalInstances_WithName PASSED
    

    It's unclear to me why these exceptions are shown so specifically. I find it confusing, and can't seem to find a way to suppress these warnings.

    ResourceWarning: unclosed file

    When running python setup.py test, mock generate a ResourceWarning: unclosed file for three tests. The test itself succeeds.

    testBasicCase2 (tests.test_configargparse.TestBasicUseCases) ...

    /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/traceback.py:220: ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/zx/y85xrx5x0lz6bgqd45lhr9kc0000gp/T/tmpqe8s6ado' mode='r' encoding='UTF-8'>
      tb.tb_frame.clear()
    ResourceWarning: Enable tracemalloc to get the object allocation traceback
    

    testBasicCase2_WithGroups (tests.test_configargparse.TestBasicUseCases) ...

    /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/traceback.py:220: ResourceWarning: unclosed file <_io.TextIOWrapper name='/var/folders/zx/y85xrx5x0lz6bgqd45lhr9kc0000gp/T/tmpwocvu164' mode='r' encoding='UTF-8'>
      tb.tb_frame.clear()
    ResourceWarning: Enable tracemalloc to get the object allocation traceback
    

    testKwrgsArePassedToArgParse (tests.test_configargparse.TestMisc) ...

    /Users/freek/Library/Python/3.8/lib/python/site-packages/mock/mock.py:2072: ResourceWarning: unclosed file <_io.FileIO name=3 mode='rb+' closefd=True>
      setattr(_type, entry, MagicProxy(entry, self))
    ResourceWarning: Enable tracemalloc to get the object allocation traceback
    /Users/freek/Library/Python/3.8/lib/python/site-packages/mock/mock.py:2072: ResourceWarning: unclosed file <_io.FileIO name=4 mode='rb+' closefd=True>
      setattr(_type, entry, MagicProxy(entry, self))
    ResourceWarning: Enable tracemalloc to get the object allocation traceback
    /Users/freek/Library/Python/3.8/lib/python/site-packages/mock/mock.py:2072: ResourceWarning: unclosed file <_io.FileIO name=5 mode='rb+' closefd=True>
      setattr(_type, entry, MagicProxy(entry, self))
    ResourceWarning: Enable tracemalloc to get the object allocation traceback
    
    opened by macfreek 7
  • Rewrite DefaultConfigFileParser.parse()

    Rewrite DefaultConfigFileParser.parse()

    This allows for empty values: key= or key="". This is different from specifying key without a value (with still sets key to "true"). Closes #103 Closes #136

    This allows for negative values, or values that start with a dash: key=-10, key=-A38E-8 Closes #137 (though that most likely was already resolved by #160)

    This allows to specify special characters, by putting them in quotes (either ' or " quotes): key=":", key="=", key="#", key="'", key='"', etc.

    Also adds test suites to explicitly test for the different values and syntaxes. Beware that by adding these test suites some explicity choices are made in particular to the allowed syntax of keys. Previously keys may contain more or less any character (at least according to the parser). ~~Now, allowed characters are: 0-9A-Za-z_-. Thus, e.g. key$name or key^name is no longer allowed.~~ Edit: This remains so in this version.

    opened by macfreek 7
  • ConfigFileParse does not handle action='append'

    ConfigFileParse does not handle action='append'

    Only the last key/value pair in the file gets stored, eg:

    --foo a --foo b --foo c

    Will produce options.foo ['a', 'b', 'c']

    But a file with:

    foo = a foo = b foo = c

    Will only produce ['c']

    opened by russdill 7
  • namespacing based on config?

    namespacing based on config?

    Thank you for your efforts. This library has a lot of what I want, especially the ability to to easily parse a input config file. But was wondering if you could let me know if and how I can accomplish the following scenario.

    Assume I have 2 or more datasets I want to process using my software and concatenate them all in the final output

    dataset1 would run with parameters set on config1 file dataset2 would run would parameters set on config2 file

    command line arguments would be common to both datasets.

    is there a way I could accomplish this?

    without it I need to run my process in 2 separate instances

    parse the arguments from config1 + command line > run process parse arguments from config2 + command line > run process

    I am trying to convert a program I created in perl and with that I could use GetOptionsFromArray(@args,) By parsing the input config files one by one and populating the @args variable and then running the perl equivilent of argparse (getOptions)

    opened by duartemolha 0
  • Update version of pydoctor.

    Update version of pydoctor.

    Not a released version yet, but the PR https://github.com/twisted/pydoctor/pull/358 should be merged soon, I hope.

    opened by tristanlatr 0
  • Add option to prevent config files from writing

    Add option to prevent config files from writing

    I have a use case where I need to call parse_known_args to finish the contruction of my ArgumentParser instance. Thus, I don't want it to just write the config and exit. This PR adds a flag ignore_write_args to the parse_known_args method. When True, this prevents the config files from running.

    If this change is desired, please advise on any additional change that would need to be made to support this feature (places where to change the docs / etc).

    (My editor also removed whitespace, I can undo that the maintainers would like to keep the diff minimal)

    opened by Erotemic 0
  • `write_config_file` saves all items coming from config file as strings

    `write_config_file` saves all items coming from config file as strings

    Resulting saved config file looks differently when argument is defined on command line or in the config file, which always uses strings for values. The code treats specifically does different things for different sources

                if source == _COMMAND_LINE_SOURCE_KEY:
                    _, existing_command_line_args = settings['']
                    for action in self._actions:
                        config_file_keys = self.get_possible_config_keys(action)
                        if config_file_keys and not action.is_positional_arg and \
                            already_on_command_line(existing_command_line_args,
                                                    action.option_strings,
                                                    self.prefix_chars):
                            value = getattr(parsed_namespace, action.dest, None)
                            if value is not None:
                                if isinstance(value, bool):
                                    value = str(value).lower()
                                config_file_items[config_file_keys[0]] = value
    
                elif source == _ENV_VAR_SOURCE_KEY:
                    for key, (action, value) in settings.items():
                        config_file_keys = self.get_possible_config_keys(action)
                        if config_file_keys:
                            value = getattr(parsed_namespace, action.dest, None)
                            if value is not None:
                                config_file_items[config_file_keys[0]] = value
                elif source.startswith(_CONFIG_FILE_SOURCE_KEY):
                    for key, (action, value) in settings.items():
                        config_file_items[key] = value
    

    That is problematic for config files management and also directly contradicts design goals declared by this project.

    Is there any particular reason for this behavior?

    Thanks

    opened by ipcoder 1
  • Are subcommands supported?

    Are subcommands supported?

    I've been trying to use ConfigArgParse together with sub-commands, but the behaviour is not what I expected...

    For example:

    import configargparse
    import yaml
    
    def main():
        ap = configargparse.ArgParser(
            default_config_files=['/etc/app/conf.d/*.conf', '~/.my_settings', 'defaults.ini'],
            config_file_parser_class=configargparse.ConfigparserConfigFileParser)
        subparsers = ap.add_subparsers(help="sub-command help", parser_class=configargparse.ArgParser, dest="command")
        p0 = subparsers.add_parser('cmd0', help="This is command 0")
        p0.add('-r', help="A random one")
        p = subparsers.add_parser('cmd1', help="This is command 1")
        p.add('-c', '--my-config', required=False, is_config_file=True, help='config file path')
        p.add('-e', '--my-config2', required=False, is_config_file=True, help='config file path')
        p.add('-b', '--bla', required=False, default="DING")
        p.add('--genome', required=False, help='path to genome file')  # this option can be set in a config file because it starts with '--'
        p.add('-v', help='verbose', action='store_true')
        p.add('-d', '--dbsnp', help='known variants .vcf', env_var='DBSNP_PATH')  # this option can be set in a config file because it starts with '--'
        # p.add('--vcf', nargs='+', help='variant file(s)', type=yaml.safe_load)
        # p.add('--vcf2', nargs='+', help='variant file(s)', type=yaml.safe_load)
        g1 = p.add_argument_group("group 1")
        g1.add('--group1_a', help="goup 1 A", env_var='GROUP1_A')
    
        options = p.parse_known_args()
    
        print(p.format_help())
        print("----------")
        print(p.format_values())    # useful for logging where different settings came fro
        print("----------")
        print(options)
    
    main()
    

    If I now call it with python3 configargparse_tests.py -h, I get:

    usage: configargparse_tests.py cmd1 [-h] [-c MY_CONFIG] [-e MY_CONFIG2]
                                        [-b BLA] [--genome GENOME] [-v] [-d DBSNP]
                                        [--group1_a GROUP1_A]
    
    optional arguments:
      -h, --help            show this help message and exit
      -c MY_CONFIG, --my-config MY_CONFIG
                            config file path
      -e MY_CONFIG2, --my-config2 MY_CONFIG2
                            config file path
      -b BLA, --bla BLA
      --genome GENOME       path to genome file
      -v                    verbose
      -d DBSNP, --dbsnp DBSNP
                            known variants .vcf [env var: DBSNP_PATH]
    
    group 1:
      --group1_a GROUP1_A   goup 1 A [env var: GROUP1_A]
    

    No mention of cmd0 anywhere. If I run with python3 configargparse_tests.py cmd0 -h, I get the exact same output...

    opened by wabiloo 1
  • Change priority order

    Change priority order

    Is there any easy (configurable) way to configure the priority order? In my own projects, I tend to have command line > config files > env vars > defaults instead of command line > env vars > config files > defaults

    opened by wabiloo 1
  • parse_known_args() got an unexpected keyword argument 'ignore_help_args'

    parse_known_args() got an unexpected keyword argument 'ignore_help_args'

    Got this error, but I can't figure out the reason. I have checked the line 457 and the parse_known_args function in the source code but they look ok. I have to delete the ignore_help_args to make it run.

     File "~/.conda/envs/bilby-td/lib/python3.9/site-packages/configargparse.py", line 457, in parse_args
        args, argv = self.parse_known_args(
    TypeError: parse_known_args() got an unexpected keyword argument 'ignore_help_args'
    
    opened by xuyumeng 3
  • Adding `config_file_parser_class=configargparse.ConfigparserConfigFileParser,`

    Adding `config_file_parser_class=configargparse.ConfigparserConfigFileParser,`

    Adding config_file_parser_class=configargparse.ConfigparserConfigFileParser, makes the help epilogue for configargparse very wierd.

    image

    Instead of

    image

    opened by ayush4921 1
  • Encoding with utf-8

    Encoding with utf-8

    Hi there, I have a problem reading a config file. Maybe I'm dumb but I can't find the solution. When opening a config file how can I specify to use utf-8 encoding instead of cp1252? Some people have trouble with not Latin characters and I found out that problem occurs inside configargparse.py/ArgumentParser/parse_known_args.. Maybe I've something to specify in the head of the config file? Any suggestions?

    opened by mastrolube 3
  • prefix matching values are not (handled) prioritized

    prefix matching values are not (handled) prioritized

    In my scenrio I have values being set in the config file like: ignore = [errors]

    Starting my script with: htcanalyze --ignore recources

    will override the internal list ignore = [recources]

    HOWEVER, when I only use the prefix: htcanalyze --ign recources

    it will result in ignore = [errors].

    In this case the config values will not be overwritten, which will result in different user experiences cause no error is thrown due to the ability of argparse on matching prefixes.

    This is related to the issue of my project: https://github.com/psyinfra/HTCAnalyze/issues/90

    opened by mathisloevenich 2
Releases(v1.5.3)
Pythonic command line arguments parser, that will make you smile

docopt creates beautiful command-line interfaces Video introduction to docopt: PyCon UK 2012: Create *beautiful* command-line interfaces with Python N

null 7.5k Oct 25, 2021
Cleo allows you to create beautiful and testable command-line interfaces.

Cleo Create beautiful and testable command-line interfaces. Cleo is mostly a higher level wrapper for CliKit, so a lot of the components and utilities

Sébastien Eustace 763 Oct 15, 2021
A thin, practical wrapper around terminal capabilities in Python

Blessings Coding with Blessings looks like this... from blessings import Terminal t = Terminal() print(t.bold('Hi there!')) print(t.bold_red_on_brig

Erik Rose 1.3k Oct 19, 2021
Typer, build great CLIs. Easy to code. Based on Python type hints.

Typer, build great CLIs. Easy to code. Based on Python type hints. Documentation: https://typer.tiangolo.com Source Code: https://github.com/tiangolo/

Sebastián Ramírez 6.5k Oct 22, 2021
Color text streams with a polished command line interface

colout(1) -- Color Up Arbitrary Command Output Synopsis colout [-h] [-r RESOURCE] colout [-g] [-c] [-l min,max] [-a] [-t] [-T DIR] [-P DIR] [-d COLORM

nojhan 1.1k Sep 28, 2021
Library for building powerful interactive command line applications in Python

Python Prompt Toolkit prompt_toolkit is a library for building powerful interactive command line applications in Python. Read the documentation on rea

prompt-toolkit 7.3k Oct 23, 2021
Cement is an advanced Application Framework for Python, with a primary focus on CLI

Cement Framework Cement is an advanced Application Framework for Python, with a primary focus on Command Line Interfaces (CLI). Its goal is to introdu

Data Folk Labs, LLC 1k Oct 25, 2021
A cross platform package to do curses-like operations, plus higher level APIs and widgets to create text UIs and ASCII art animations

ASCIIMATICS Asciimatics is a package to help people create full-screen text UIs (from interactive forms to ASCII animations) on any platform. It is li

null 2.8k Oct 23, 2021
Rich is a Python library for rich text and beautiful formatting in the terminal.

Rich 中文 readme • lengua española readme • Läs på svenska Rich is a Python library for rich text and beautiful formatting in the terminal. The Rich API

Will McGugan 30.4k Oct 24, 2021
Python Command-line Application Tools

Clint: Python Command-line Interface Tools Clint is a module filled with a set of awesome tools for developing commandline applications. C ommand L in

Kenneth Reitz Archive 57 Oct 12, 2021
sane is a command runner made simple.

sane is a command runner made simple.

Miguel M. 14 Jul 24, 2021
CalcuPy 📚 Create console-based calculators in a few lines of code.

CalcuPy ?? Create console-based calculators in a few lines of code. ?? Installation pip install calcupy ?? Usage from calcupy import Calculator calc

Dylan Tintenfich 6 Sep 26, 2021
Python library that measures the width of unicode strings rendered to a terminal

Introduction This library is mainly for CLI programs that carefully produce output for Terminals, or make pretend to be an emulator. Problem Statement

Jeff Quast 250 Oct 15, 2021
Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.

Python Fire Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object. Python Fire is a s

Google 20.3k Oct 24, 2021
A fast, stateless http slash commands framework for scale. Built by the Crunchy bot team.

Roid ?? A fast, stateless http slash commands framework for scale. Built by the Crunchy bot team. ?? Installation You can install roid in it's default

Harrison Burt 7 Oct 17, 2021
Simple cross-platform colored terminal text in Python

Colorama Makes ANSI escape character sequences (for producing colored terminal text and cursor positioning) work under MS Windows. PyPI for releases |

Jonathan Hartley 2.6k Oct 23, 2021
A simple terminal Christmas tree made with Python

Python Christmas Tree A simple CLI Christmas tree made with Python Installation Just clone the repository and run $ python terminal_tree.py More opti

Francisco B. 34 Oct 6, 2021
emoji terminal output for Python

Emoji Emoji for Python. This project was inspired by kyokomi. Example The entire set of Emoji codes as defined by the unicode consortium is supported

Taehoon Kim 1.3k Oct 16, 2021
plotting in the terminal

bashplotlib plotting in the terminal what is it? bashplotlib is a python package and command line tool for making basic plots in the terminal. It's a

Greg Lamp 1.6k Oct 17, 2021