dict subclass with keylist/keypath support, normalized I/O operations (base64, csv, ini, json, pickle, plist, query-string, toml, xml, yaml) and many utilities.

Overview

Code style: black

python-benedict

python-benedict is a dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, ini, json, pickle, plist, query-string, toml, xml, yaml) and many utilities... for humans, obviously.

Features

  • 100% backward-compatible, you can safely wrap existing dictionaries.
  • Keylist support using list of keys as key.
  • Keypath support using keypath-separator (dot syntax by default).
  • Keypath list-index support (also negative) using the standard [n] suffix.
  • Normalized I/O operations with most common formats: base64, csv, ini, json, pickle, plist, query-string, toml, xml, yaml.
  • Many utility and parse methods to retrieve data as needed (check the API section).
  • Well tested. ;)

Index

Installation

  • Run pip install python-benedict

Usage

Basics

benedict is a dict subclass, so it is possible to use it as a normal dictionary (you can just cast an existing dict).

from benedict import benedict

# create a new empty instance
d = benedict()

# or cast an existing dict
d = benedict(existing_dict)

# or create from data source (filepath, url or data-string) in a supported format:
# Base64, CSV, JSON, TOML, XML, YAML, query-string
d = benedict('https://localhost:8000/data.json', format='json')

# or in a Django view
params = benedict(request.GET.items())
page = params.get_int('page', 1)

Keylist

Wherever a key is used, it is possible to use also a list (or a tuple) of keys.

d = benedict()

# set values by keys list
d['profile', 'firstname'] = 'Fabio'
d['profile', 'lastname'] = 'Caccamo'
print(d) # -> { 'profile':{ 'firstname':'Fabio', 'lastname':'Caccamo' } }
print(d['profile']) # -> { 'firstname':'Fabio', 'lastname':'Caccamo' }

# check if keypath exists in dict
print(['profile', 'lastname'] in d) # -> True

# delete value by keys list
del d['profile', 'lastname']
print(d['profile']) # -> { 'firstname':'Fabio' }

Keypath

. is the default keypath separator.

If you cast an existing dict and its keys contain the keypath separator a ValueError will be raised.

In this case you should use a custom keypath separator or disable keypath functionality.

d = benedict()

# set values by keypath
d['profile.firstname'] = 'Fabio'
d['profile.lastname'] = 'Caccamo'
print(d) # -> { 'profile':{ 'firstname':'Fabio', 'lastname':'Caccamo' } }
print(d['profile']) # -> { 'firstname':'Fabio', 'lastname':'Caccamo' }

# check if keypath exists in dict
print('profile.lastname' in d) # -> True

# delete value by keypath
del d['profile.lastname']

Custom keypath separator

You can customize the keypath separator passing the keypath_separator argument in the constructor.

If you pass an existing dict to the constructor and its keys contain the keypath separator an Exception will be raised.

d = benedict(existing_dict, keypath_separator='/')

Change keypath separator

You can change the keypath_separator at any time using the getter/setter property.

If any existing key contains the new keypath_separator an Exception will be raised.

d.keypath_separator = '/'

Disable keypath functionality

You can disable the keypath functionality passing keypath_separator=None in the constructor.

d = benedict(existing_dict, keypath_separator=None)

You can disable the keypath functionality using the getter/setter property.

d.keypath_separator = None

List index support

List index are supported, keypaths can include indexes (also negative) using [n], to perform any operation very fast:

# Eg. get last location cordinates of the first result:
loc = d['results[0].locations[-1].coordinates']
lat = loc.get_decimal('latitude')
lng = loc.get_decimal('longitude')

API

Utility methods

These methods are common utilities that will speed up your everyday work.

Utilities that accept key argument(s) also support keypath(s).

Utilities that return a dictionary always return a new benedict instance.

  • clean

# Clean the current dict instance removing all empty values: None, '', {}, [], ().
# If strings or collections (dict, list, set, tuple) flags are False,
# related empty values will not be deleted.
d.clean(strings=True, collections=True)
  • clone

# Return a clone (deepcopy) of the dict.
c = d.clone()
  • dump

# Return a readable representation of any dict/list.
# This method can be used both as static method or instance method.
s = benedict.dump(d.keypaths())
print(s)
# or
d = benedict()
print(d.dump())
  • filter

# Return a filtered dict using the given predicate function.
# Predicate function receives key, value arguments and should return a bool value.
predicate = lambda k, v: v is not None
f = d.filter(predicate)
  • find

# Return the first match searching for the given keys/keypaths.
# If no result found, default value is returned.
keys = ['a.b.c', 'm.n.o', 'x.y.z']
f = d.find(keys, default=0)
  • flatten

# Return a new flattened dict using the given separator to join nested dict keys to flatten keypaths.
f = d.flatten(separator='_')
  • groupby

# Group a list of dicts at key by the value of the given by_key and return a new dict.
g = d.groupby('cities', by_key='country_code')
  • invert

# Return an inverted dict where values become keys and keys become values.
# Since multiple keys could have the same value, each value will be a list of keys.
# If flat is True each value will be a single value (use this only if values are unique).
i = d.invert(flat=False)
  • items_sorted_by_keys

# Return items (key/value list) sorted by keys.
# If reverse is True, the list will be reversed.
items = d.items_sorted_by_keys(reverse=False)
  • items_sorted_by_values

# Return items (key/value list) sorted by values.
# If reverse is True, the list will be reversed.
items = d.items_sorted_by_values(reverse=False)
  • keypaths

# Return a list of all keypaths in the dict.
# If indexes is True, the output will include list values indexes.
k = d.keypaths(indexes=False)
  • match

# Return a list of all values whose keypath matches the given pattern (a regex or string).
# If pattern is string, wildcard can be used (eg. [*] can be used to match all list indexes).
# If indexes is True, the pattern will be matched also against list values.
m = d.match(pattern, indexes=True)
  • merge

# Merge one or more dictionary objects into current instance (deepupdate).
# Sub-dictionaries keys will be merged toghether.
# If overwrite is False, existing values will not be overwritten.
# If concat is True, list values will be concatenated toghether.
d.merge(a, b, c, overwrite=True, concat=False)
  • move

# Move an item from key_src to key_dst.
# It can be used to rename a key.
# If key_dst exists, its value will be overwritten.
d.move('a', 'b', overwrite=True)
  • nest

# Nest a list of dicts at the given key and return a new nested list
# using the specified keys to establish the correct items hierarchy.
d.nest('values', id_key='id', parent_id_key='parent_id', children_key='children')
  • remove

# Remove multiple keys from the dict.
# It is possible to pass a single key or more keys (as list or *args).
d.remove(['firstname', 'lastname', 'email'])
  • rename

# Rename a dict item key from 'key' to 'key_new'.
# If key_new exists, a KeyError will be raised.
d.rename('first_name', 'firstname')
  • search

# Search and return a list of items (dict, key, value, ) matching the given query.
r = d.search('hello', in_keys=True, in_values=True, exact=False, case_sensitive=False)
  • standardize

"location_latitude". d.standardize()">
# Standardize all dict keys, e.g. "Location Latitude" -> "location_latitude".
d.standardize()
  • subset

# Return a dict subset for the given keys.
# It is possible to pass a single key or more keys (as list or *args).
s = d.subset(['firstname', 'lastname', 'email'])
  • swap

# Swap items values at the given keys.
d.swap('firstname', 'lastname')
  • traverse

# Traverse a dict passing each item (dict, key, value) to the given callback function.
def f(d, key, value):
    print('dict: {} - key: {} - value: {}'.format(d, key, value))
d.traverse(f)
  • unflatten

# Return a new unflattened dict using the given separator to split dict keys to nested keypaths.
u = d.unflatten(separator='_')
  • unique

# Remove duplicated values from the dict.
d.unique()

I/O methods

It is possible to create a benedict instance directly from data source (filepath, url or data-string) by passing the data source and the data format (default 'json') in the constructor.

# filepath
d = benedict('/root/data.yml', format='yaml')

# url
d = benedict('https://localhost:8000/data.xml', format='xml')

# data-string
d = benedict('{"a": 1, "b": 2, "c": 3, "x": 7, "y": 8, "z": 9}')

These methods simplify I/O operations with most common formats: base64, csv, json, pickle, plist, query-string, toml, xml, yaml.

In all from_* methods, the first argument can be: url, filepath or data-string.

In all to_* methods, if filepath='...' kwarg is specified, the output will be also saved at the specified filepath.

  • from_base64

# Try to load/decode a base64 encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to choose the subformat used under the hood:
# (`csv`, `json`, `query-string`, `toml`, `xml`, `yaml`), default: 'json'.
# It's possible to choose the encoding, default 'utf-8'.
# A ValueError is raised in case of failure.
d = benedict.from_base64(s, subformat='json', encoding='utf-8', **kwargs)
  • from_csv

# Try to load/decode a csv encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to specify the columns list, default: None (in this case the first row values will be used as keys).
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/csv.html
# A ValueError is raised in case of failure.
d = benedict.from_csv(s, columns=None, columns_row=True, **kwargs)
  • from_ini

# Try to load/decode a ini encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/configparser.html
# A ValueError is raised in case of failure.
d = benedict.from_ini(s, **kwargs)
  • from_json

# Try to load/decode a json encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/json.html
# A ValueError is raised in case of failure.
d = benedict.from_json(s, **kwargs)
  • from_pickle

# Try to load/decode a pickle encoded in Base64 format and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/pickle.html
# A ValueError is raised in case of failure.
d = benedict.from_pickle(s, **kwargs)
  • from_plist

# Try to load/decode a p-list encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/plistlib.html
# A ValueError is raised in case of failure.
d = benedict.from_plist(s, **kwargs)
  • from_query_string

# Try to load/decode a query-string and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# A ValueError is raised in case of failure.
d = benedict.from_query_string(s, **kwargs)
  • from_toml

# Try to load/decode a toml encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://pypi.org/project/toml/
# A ValueError is raised in case of failure.
d = benedict.from_toml(s, **kwargs)
  • from_xml

# Try to load/decode a xml encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://github.com/martinblech/xmltodict
# A ValueError is raised in case of failure.
d = benedict.from_xml(s, **kwargs)
  • from_yaml

# Try to load/decode a yaml encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://pyyaml.org/wiki/PyYAMLDocumentation
# A ValueError is raised in case of failure.
d = benedict.from_yaml(s, **kwargs)
  • to_base64

# Return the dict instance encoded in base64 format and optionally save it at the specified 'filepath'.
# It's possible to choose the subformat used under the hood:
# ('csv', json', `query-string`, 'toml', 'xml', 'yaml'), default: 'json'.
# It's possible to choose the encoding, default 'utf-8'.
# It's possible to pass decoder specific options using kwargs.
# A ValueError is raised in case of failure.
s = d.to_base64(subformat='json', encoding='utf-8', **kwargs)
  • to_csv

# Return a list of dicts in the current dict encoded in csv format and optionally save it at the specified filepath.
# It's possible to specify the key of the item (list of dicts) to encode, default: 'values'.
# It's possible to specify the columns list, default: None (in this case the keys of the first item will be used).
# A ValueError is raised in case of failure.
s = d.to_csv(key='values', columns=None, columns_row=True, **kwargs)
  • to_ini

# Return the dict instance encoded in ini format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/configparser.html
# A ValueError is raised in case of failure.
s = d.to_ini(**kwargs)
  • to_json

# Return the dict instance encoded in json format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/json.html
# A ValueError is raised in case of failure.
s = d.to_json(**kwargs)
  • to_pickle

# Return the dict instance as pickle encoded in Base64 format and optionally save it at the specified filepath.
# The pickle protocol used by default is 2.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/pickle.html
# A ValueError is raised in case of failure.
s = d.to_pickle(**kwargs)
  • to_plist

# Return the dict instance encoded in p-list format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/plistlib.html
# A ValueError is raised in case of failure.
s = d.to_plist(**kwargs)
  • to_query_string

# Return the dict instance as query-string and optionally save it at the specified filepath.
# A ValueError is raised in case of failure.
s = d.to_query_string(**kwargs)
  • to_toml

# Return the dict instance encoded in toml format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://pypi.org/project/toml/
# A ValueError is raised in case of failure.
s = d.to_toml(**kwargs)
  • to_xml

# Return the dict instance encoded in xml format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://github.com/martinblech/xmltodict
# A ValueError is raised in case of failure.
s = d.to_xml(**kwargs)
  • to_yaml

# Return the dict instance encoded in yaml format.
# If filepath option is passed the output will be saved ath
# It's possible to pass encoder specific options using kwargs:
# https://pyyaml.org/wiki/PyYAMLDocumentation
# A ValueError is raised in case of failure.
s = d.to_yaml(**kwargs)

Parse methods

These methods are wrappers of the get method, they parse data trying to return it in the expected type.

  • get_bool

# Get value by key or keypath trying to return it as bool.
# Values like `1`, `true`, `yes`, `on`, `ok` will be returned as `True`.
d.get_bool(key, default=False)
  • get_bool_list

# Get value by key or keypath trying to return it as list of bool values.
# If separator is specified and value is a string it will be splitted.
d.get_bool_list(key, default=[], separator=',')
  • get_date

# Get value by key or keypath trying to return it as date.
# If format is not specified it will be autodetected.
# If choices and value is in choices return value otherwise default.
d.get_date(key, default=None, format=None, choices=[])
  • get_date_list

# Get value by key or keypath trying to return it as list of date values.
# If separator is specified and value is a string it will be splitted.
d.get_date_list(key, default=[], format=None, separator=',')
  • get_datetime

# Get value by key or keypath trying to return it as datetime.
# If format is not specified it will be autodetected.
# If choices and value is in choices return value otherwise default.
d.get_datetime(key, default=None, format=None, choices=[])
  • get_datetime_list

# Get value by key or keypath trying to return it as list of datetime values.
# If separator is specified and value is a string it will be splitted.
d.get_datetime_list(key, default=[], format=None, separator=',')
  • get_decimal

# Get value by key or keypath trying to return it as Decimal.
# If choices and value is in choices return value otherwise default.
d.get_decimal(key, default=Decimal('0.0'), choices=[])
  • get_decimal_list

# Get value by key or keypath trying to return it as list of Decimal values.
# If separator is specified and value is a string it will be splitted.
d.get_decimal_list(key, default=[], separator=',')
  • get_dict

# Get value by key or keypath trying to return it as dict.
# If value is a json string it will be automatically decoded.
d.get_dict(key, default={})
  • get_email

# Get email by key or keypath and return it.
# If value is blacklisted it will be automatically ignored.
# If check_blacklist is False, it will be not ignored even if blacklisted.
d.get_email(key, default='', choices=None, check_blacklist=True)
  • get_float

# Get value by key or keypath trying to return it as float.
# If choices and value is in choices return value otherwise default.
d.get_float(key, default=0.0, choices=[])
  • get_float_list

# Get value by key or keypath trying to return it as list of float values.
# If separator is specified and value is a string it will be splitted.
d.get_float_list(key, default=[], separator=',')
  • get_int

# Get value by key or keypath trying to return it as int.
# If choices and value is in choices return value otherwise default.
d.get_int(key, default=0, choices=[])
  • get_int_list

# Get value by key or keypath trying to return it as list of int values.
# If separator is specified and value is a string it will be splitted.
d.get_int_list(key, default=[], separator=',')
  • get_list

# Get value by key or keypath trying to return it as list.
# If separator is specified and value is a string it will be splitted.
d.get_list(key, default=[], separator=',')
  • get_list_item

# Get list by key or keypath and return value at the specified index.
# If separator is specified and list value is a string it will be splitted.
d.get_list_item(key, index=0, default=None, separator=',')
  • get_phonenumber

# Get phone number by key or keypath and return a dict with different formats (e164, international, national).
# If country code is specified (alpha 2 code), it will be used to parse phone number correctly.
d.get_phonenumber(key, country_code=None, default=None)
  • get_slug

# Get value by key or keypath trying to return it as slug.
# If choices and value is in choices return value otherwise default.
d.get_slug(key, default='', choices=[])
  • get_slug_list

# Get value by key or keypath trying to return it as list of slug values.
# If separator is specified and value is a string it will be splitted.
d.get_slug_list(key, default=[], separator=',')
  • get_str

# Get value by key or keypath trying to return it as string.
# Encoding issues will be automatically fixed.
# If choices and value is in choices return value otherwise default.
d.get_str(key, default='', choices=[])
  • get_str_list

# Get value by key or keypath trying to return it as list of str values.
# If separator is specified and value is a string it will be splitted.
d.get_str_list(key, default=[], separator=',')
  • get_uuid

# Get value by key or keypath trying to return it as valid uuid.
# If choices and value is in choices return value otherwise default.
d.get_uuid(key, default='', choices=[])
  • get_uuid_list

# Get value by key or keypath trying to return it as list of valid uuid values.
# If separator is specified and value is a string it will be splitted.
d.get_uuid_list(key, default=[], separator=',')

Testing

# create python virtual environment
virtualenv testing_benedict

# activate virtualenv
cd testing_benedict && . bin/activate

# clone repo
git clone https://github.com/fabiocaccamo/python-benedict.git src && cd src

# install requirements
pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-test.txt

# run tests using tox
tox

# or run tests using unittest
python -m unittest

# or run tests using setuptools
python setup.py test

License

Released under MIT License.


See also

Comments
  • Cannort import benedict

    Cannort import benedict

    Python version 3.8

    Package version 0.22.2

    Current behavior (bug description)

    from benedict import benedict Traceback (most recent call last): File "", line 1, in ImportError: cannot import name 'benedict' from 'benedict' (unknown location)

    Expected behavior It should be possible to import and use benedict.

    ** Additional pip information $ pip show python-benedict Name: python-benedict Version: 0.22.2 Summary: python-benedict is a dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, json, pickle, plist, query-string, toml, xml, yaml) and many utilities... for humans, obviously. Home-page: https://github.com/fabiocaccamo/python-benedict Author: Fabio Caccamo Author-email: [email protected] License: MIT

    invalid 
    opened by gschoenberger 11
  • The packages fails to load AWS SAM yaml format with resources reference

    The packages fails to load AWS SAM yaml format with resources reference

    Python version 3.8

    Package version @latest (10.08.20)

    Current behavior (bug description)

    Loading a YAML template with the following entry

    ...
    AccountsFunction:
        Description: "Accounts Function ARN"
        Value: !GetAtt AccountsLambda.Arn
    

    throws the following exception

    ValueError: Invalid data or url or filepath argument: .../template.yaml
    could not determine a constructor for the tag '!GetAtt'
    

    Expected behavior Such tags should be supported as well

    I can't provide the whole template as it contains sensitive info but the provided info should be sufficient. Please contact me if you have further questions. Thanks in advance.

    invalid 
    opened by godrose 10
  • wrong key-values updates using pointers

    wrong key-values updates using pointers

    Python version Python 3.6.9

    Package version python-benedict (0.18.1)

    Current behavior (bug description) I'll try to explain the problem using a simple example.

    `my_servers="""

    SERVER:
        S01:
            alias: s01_alias
            ssh_port: s01_port
            host: s01_host
            credentials:
                username: s01_user
                password: s01_passw
            location:
                building: s01_building
                floar: s01_floar
                room: s01_room
    

    `

    Creating a benedict dictionary:

    servers=benedict.from_yaml(my_servers)
    

    Creating a couple of pointers (are they pointers???)

    s01_ptr=benedict(servers['SERVER.S01'])
    s01_copy=benedict(servers['SERVER.S01'].copy())
    

    Changing some items:

     s01_ptr['alias']='ptr_alias'
     s01_ptr['location.building']='ptr_building'
     s01_ptr['credentials.username']='ptr_unsername'
    

    OUTPUT of the three objects:

    --- s01_ptr:
    {
        "alias": "ptr_alias",    <--- changed
        "credentials": {
            "password": "s01_passw",
            "username": "ptr_unsername"    <--- changed
        },
        "host": "s01_host",
        "location": {
            "building": "ptr_building",    <--- changed
            "floar": "s01_floar",
            "room": "s01_room"
        },
        "ssh_port": "s01_port"
    }
    
    --- s01_copy:
    {
        "alias": "s01_alias",    <--- NOT changed
        "credentials": {
            "password": "s01_passw",
            "username": "ptr_unsername"    <--- changed
        },
        "host": "s01_host",
        "location": {
            "building": "ptr_building",    <--- changed
            "floar": "s01_floar",
            "room": "s01_room"
        },
        "ssh_port": "s01_port"
    }
    
    --- servers:
    {
        "SERVER": {
            "S01": {
                "alias": "s01_alias",    <--- NOT changed
                "credentials": {
                    "password": "s01_passw",
                    "username": "ptr_unsername"    <--- changed
                },
                "host": "s01_host",
                "location": {
                    "building": "ptr_building",    <--- changed
                    "floar": "s01_floar",
                    "room": "s01_room"
                },
                "ssh_port": "s01_port"
            }
        }
    }
    

    Expected behavior As we can see, on the output, all objects are impacted by the changes: s01_ptr has been affected by all changes. It's OK s01_copy only the sub-keys out of the root have been affected. It's NOT OK original server object just as s01_copy. It's NOT OK

    So if they are pointers, the result is wrong, if they are copies the result is wrong.

    Obviously, all this in case I have not made a huge mistake, in which case, I apologize and in any case thank you for the time spent analyzing the problem.

    Just another question: what are the differences between the two statements used to create s01_ptr and s01_copy? For as I see it, s01_ptr should return a real pointer (just as in a native python dictionary) while s01_copy should return a copy according to the .copy() suffix. At the end what are the differences between the copy() and clone().

    Thank You for Your time Loreto

    P.S.: Attached a simple script to re-create the problem. Benedict_test02.zip

    invalid 
    opened by Loreton 9
  • Performance issue

    Performance issue

    Python version 3.7

    Package version 0.18.0 vs 0.21.0

    Current behavior (bug description) 0.21.0 is up to 30 times slower in some instances than 0.18.0 for "large" dictionaries

    Expected behavior Performance is equal or better in 0.21.0

    MicrosoftTeams-image (2)

    Code to reproduce: import benedict from benedict import benedict as ben import logging import os logger = logging.getLogger(os.path.basename(file)) logger.setLevel(logging.INFO)

    formatter = logging.Formatter('%(asctime)s : [%(levelname)s] : %(name)s : %(message)s') log_file_handler = logging.StreamHandler() log_file_handler.setFormatter(formatter) logger.addHandler(log_file_handler)

    Create large nested dictionary

    test = ben({})

    logger.info(f"Starting test with python-benedict version: {benedict.version}") for i in range(0, 500): for j in range(0, 100): test.set(f"{i}.{j}", f"text-{i}-{j}")

    Access multiple elements with a few missing element paths

    for i in range(0, 550): for j in range(0, 110): test.get(f"{i}.{j}", None)

    logger.info("End")

    bug 
    opened by alexespencer 8
  • TypeError: 'module' object is not callable

    TypeError: 'module' object is not callable

    Python version 3.6.9

    Package version 0.18.1

    Current behavior (bug description) on the very first line where i call the benedict() class, an exception is thrown:

    Traceback (most recent call last):
      File "benedict-test.py", line 18, in <module>
        d = benedict()
    TypeError: 'module' object is not callable
    
    

    This even happens when creating a dictionary just as your usage examples demonstrate.

    Expected behavior see above!

    invalid 
    opened by hagfelsh 8
  • List indexes in keypath?

    List indexes in keypath?

    Hi there, I'm loving benedict, but have a feature request... Can we have something like this:

    from benedict import benedict
    
    d = benedict({
        "a": [
            {"b": 42}, 
            {"b": 24}
        ]
    })
    d.get("a.1.b")
    => 24
    
    enhancement 
    opened by tadams42 8
  • Why this solves all the issues regarding JSOn encoding?

    Why this solves all the issues regarding JSOn encoding?

    https://github.com/fabiocaccamo/python-benedict/blob/92d9a32d5b214839c35a63b71bae21f7864a3535/benedict/dicts/init.py#L285

    Question: how come this solves the issue? Curious and want to learn. Thank you πŸ™

    I understand that cpython has this

    https://github.com/python/cpython/blob/15f0a45b2822607f2c38685a72d06f050e24b215/Lib/json/encoder.py#L246

    if (_one_shot and c_make_encoder is not None
                    and self.indent is None):
                _iterencode = c_make_encoder(
                    markers, self.default, _encoder, self.indent,
                    self.key_separator, self.item_separator, self.sort_keys,
                    self.skipkeys, self.allow_nan)
            else:
                _iterencode = _make_iterencode(
                    markers, self.default, _encoder, self.indent, floatstr,
                    self.key_separator, self.item_separator, self.sort_keys,
                    self.skipkeys, _one_shot)
            return _iterencode(o, 0)
    

    so i guess the question is what's the difference between _make_iterencode and c_make_encoder? Performance issues?

    question 
    opened by simkimsia 7
  • json.dumps no longer works directly with benedict in 0.20.0

    json.dumps no longer works directly with benedict in 0.20.0

    Python version 3.7.7

    Package version 0.20.0

    Django version 2.2.15

    Current behavior (bug description)

    Let's say you have a Django 2.2 model that uses the postgres JSONField

    from django.contrib.postgres.fields import JSONField
    from django.db import models
    
    class SomeModel(models.Model):
        linkage = JSONField(default=dict)
    

    When you then do

    normal_dict = {'list': {'scope': {'pk': '109'}}
    benedict_dict = benedict(normal_dict)
    
    instance = SomeModel.objects.create(linkage=benedict_dict)
    retrieved =SomeModel.objects.last() 
    

    the instance.linkage will definitely still show {'list': {'scope': {'pk': '109'}} but retrieved.linkage will definitely show {}

    What if cast to dict?

    if we do this

    normal_dict = {'list': {'scope': {'pk': '109'}}
    benedict_dict = benedict(normal_dict)
    
    instance = SomeModel.objects.create(linkage=dict(benedict_dict))
    
    

    the instance.linkage will definitely still show {'list': {'scope': {'pk': '109'}} but retrieved.linkage will definitely show {'list':{}}

    My workaround

    
    normal_dict = {'list': {'scope': {'pk': '109'}}
    benedict_dict = benedict(normal_dict)
    
    sanitized_dict = benedict_dict
    if isinstance(benedict_dict, benedict):
        sanitized_dict = json.loads(benedict_dict.to_json())
    
    instance = SomeModel.objects.create(linkage=sanitized_dict)
    instance = SomeModel.objects.create(linkage=sanitized_dict)
    

    Expected behavior in 0.19.0, this situation does not occur.

    But in 0.20.0 this situation occurs. Based on my reading of the changelog it appears that all the methods will auto cast everything to benedict. I guess this is the issue.

    Also I would say this goes against the spirit of the line in the readme https://github.com/fabiocaccamo/python-benedict#basics where it says you can use benedict as a normal dict. That's not true anymore with regards with 0.20.0 and JSONField

    bug 
    opened by simkimsia 7
  • [Question] Can I use [n] as a way to loop through all in list?

    [Question] Can I use [n] as a way to loop through all in list?

    In [1]:from benedict import benedict
    
    In [2]: test = {"lorem": [{"ipsum":"a"}, {"ipsum": "b"}]}
    
    In [3]: benedict_test = benedict(test)
    
    In [4]: result = benedict_test.get("lorem[0].ipsum")
    
    In [5]: result
    Out[5]: 'a'
    
    In [6]: result = benedict_test.get("lorem[n].ipsum")
    
    In [7]: result
    

    I guess the answer to my question is a simple no.

    But I was wondering if there's a way to get what I want? Either via a feature request or via other pythonic code to work with existing benedict?

    Expected output is a list of ['a', 'b']

    enhancement 
    opened by simkimsia 7
  • to_toml() ValueError: Circular reference detected

    to_toml() ValueError: Circular reference detected

    Python version

    3.7.0

    Package version

    0.23.2

    Current behavior (bug description)

    After reading and modifying a TOML value in a benedict object. If trying to write the changes back to the TOML a ValueError is raised.

    Example:

    from benedict import benedict
    pyproject_toml = benedict("./pyproject.toml", format="toml")
    pyproject_toml["tool.poetry.name"] = "name"
    pyproject_toml.to_toml()
    

    Execption raised:

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/.../.venv/lib/python3.7/site-packages/benedict/dicts/io/io_dict.py", line 223, in to_toml
        return self._encode(self, 'toml', **kwargs)
      File "/.../.venv/lib/python3.7/site-packages/benedict/dicts/io/io_dict.py", line 55, in _encode
        s = io_util.encode(d, format, **kwargs)
      File "/.../.venv/lib/python3.7/site-packages/benedict/dicts/io/io_util.py", line 28, in encode
        s = serializer.encode(d, **kwargs)
      File "/.../.venv/lib/python3.7/site-packages/benedict/serializers/toml.py", line 20, in encode
        data = toml.dumps(d, **kwargs)
      File "/.../.venv/lib/python3.7/site-packages/toml/encoder.py", line 67, in dumps
        raise ValueError("Circular reference detected")
    ValueError: Circular reference detected
    

    Expected behavior

    New changed TOML dumped to stdout as string and if filepath argument specified, write to destination filepath.

    The bug is identified in the code and a PR to fix the problem is going to be submitted ASAP.

    bug 
    opened by next-franciscoalgaba 6
  • filter and modify list of dicts

    filter and modify list of dicts

    So I have a list of dicts like this (this dict could be in nested dict as well): guilds = [{"ID":1, "Name":"x"}, {"ID":2, "Name":"y"}]

    I want to modify the value of the Name key in the dict which ID is 1. I could not find how can I do that with the library.

    enhancement 
    opened by quancore 5
  • Bump coverage from 7.0.1 to 7.0.2

    Bump coverage from 7.0.1 to 7.0.2

    Bumps coverage from 7.0.1 to 7.0.2.

    Changelog

    Sourced from coverage's changelog.

    Version 7.0.2 β€” 2023-01-02

    • Fix: when using the [run] relative_files = True setting, a relative [paths] pattern was still being made absolute. This is now fixed, closing issue 1519_.

    • Fix: if Python doesn't provide tomllib, then TOML configuration files can only be read if coverage.py is installed with the [toml] extra. Coverage.py will raise an error if TOML support is not installed when it sees your settings are in a .toml file. But it didn't understand that [tools.coverage] was a valid section header, so the error wasn't reported if you used that header, and settings were silently ignored. This is now fixed, closing issue 1516_.

    • Fix: adjusted how decorators are traced on PyPy 7.3.10, fixing issue 1515_.

    • Fix: the coverage lcov report did not properly implement the --fail-under=MIN option. This has been fixed.

    • Refactor: added many type annotations, including a number of refactorings. This should not affect outward behavior, but they were a bit invasive in some places, so keep your eyes peeled for oddities.

    • Refactor: removed the vestigial and long untested support for Jython and IronPython.

    .. _issue 1515: nedbat/coveragepy#1515 .. _issue 1516: nedbat/coveragepy#1516 .. _issue 1519: nedbat/coveragepy#1519

    .. _changes_7-0-1:

    Commits
    • 2f731e2 docs: sample HTML
    • dbbd5b7 docs: prep for 7.0.2
    • d08e6d0 fix: relative_files should keep relative path maps. #1519
    • 3f0bce2 mypy: partial debug.py and pytracer.py
    • ffc701a mypy: test_xml.py
    • 5580cf8 mypy: xmlreport.py
    • 0c9b5e0 mypy: check collector.py and plugin_support.py
    • 8f4d404 refactor: a better way to filter coverage debug pybehave
    • a3f3841 mypy: add cmdline.py and test_cmdline.py
    • 09f9188 mypy: add env.py
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 1
  • Bump tox from 4.1.2 to 4.1.3

    Bump tox from 4.1.2 to 4.1.3

    Bumps tox from 4.1.2 to 4.1.3.

    Release notes

    Sourced from tox's releases.

    4.1.3

    What's Changed

    Full Changelog: https://github.com/tox-dev/tox/compare/4.1.2...4.1.3

    Changelog

    Sourced from tox's changelog.

    v4.1.3 (2023-01-02)

    Bugfixes - 4.1.3

    - Reuse package_env with ``--installpkg`` - by :user:`q0w`. (:issue:`2442`)
    - Fail more gracefully when pip :ref:`install_command` is empty - by :user:`jayaddison`. (:issue:`2695`)
    

    Improved Documentation - 4.1.3

    • Add breaking-change documentation for empty install_command values - by :user:jayaddison. (:issue:2695)

    Misc - 4.1.3

    - :issue:`2796`, :issue:`2797`
    
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies python 
    opened by dependabot[bot] 1
  • Separate installation targets for certain dependencies

    Separate installation targets for certain dependencies

    Hey there, this looks like a really useful utility. Just a recommendation -- the base install may be kept more lightweight if you made some separate install targets for certain features. For example, a pip install python-benedict[s3] option that would pull in boto.

    enhancement 
    opened by totalhack 0
  • Wildcard support for renaming

    Wildcard support for renaming

    Desired functionality as following:

    d = {
    "a": [
      {
        "x": 1,
        "y": 1,
      },
      {
        "x": 2,
        "y": 2,
      },
      ]
    }
    b = benedict(d)
    b.rename("a[*].x", "a[*].m")
    
    
    enhancement 
    opened by milhauzindahauz 9
Releases(0.28.1)
  • 0.28.1(Jan 3, 2023)

    • Add pyupgrade to pre-commit config.
    • Add setup.cfg (setuptools declarative syntax) generated using setuptools-py2cfg.
    • Add support for pathlib.Path. #144
    • Bump requirements.
    Source code(tar.gz)
    Source code(zip)
  • 0.28.0(Dec 29, 2022)

    • Drop Python 3.7 support.
    • Replace str.format with f-strings.
    • Remove python setup.py test usage.
    • Remove encoding pragma.
    • Fix s3_options option forwarded to json decoder. #198 (#204)
    • Bump requirements.
    Source code(tar.gz)
    Source code(zip)
  • 0.27.1(Nov 26, 2022)

    • Add Python 3.11 support. #143
    • Add pre-commit with black, isort and flake8.
    • Read toml files using the standard tomlib (if available). #143
    • Bump requirements (boto3, python-slugify, orjson) version.
    Source code(tar.gz)
    Source code(zip)
  • 0.27.0(Oct 12, 2022)

  • 0.26.0(Oct 9, 2022)

  • 0.25.4(Sep 6, 2022)

  • 0.25.3(Aug 23, 2022)

  • 0.25.2(Jul 15, 2022)

  • 0.25.1(Apr 27, 2022)

    • Fixed broken yaml serialization with benedict attributes. #89
    • Fixed flatten not working when separator is equal to keypath_separator. #88
    • Bumped requirements.
    Source code(tar.gz)
    Source code(zip)
  • 0.25.0(Feb 17, 2022)

    • Added official python 3.10 support.
    • Dropped python 2.7 and python 3.5 support.
    • Pinned requirements versions.
    • Reformatted code with Black.
    Source code(tar.gz)
    Source code(zip)
  • 0.24.3(Oct 4, 2021)

  • 0.24.2(Aug 11, 2021)

  • 0.24.1(Aug 1, 2021)

  • 0.24.0(May 4, 2021)

    • Added ini format support. #36 #40
    • Added python 3.9 to CI (tox, travis and GitHub actions).
    • Fixed to_toml circular reference error. #53
    • Updated ftfy requirement version depending on python version.
    • Updated (improved) QueryStringSerializer regex.
    Source code(tar.gz)
    Source code(zip)
  • 0.23.2(Jan 19, 2021)

  • 0.23.1(Jan 14, 2021)

  • 0.23.0(Dec 21, 2021)

  • 0.22.4(Dec 22, 2020)

  • 0.22.3(Dec 22, 2020)

    • Added concat option to merge method. #45
    • Added sort_keys=True by default in JSON serializer.
    • Added memo option to clone core method.
    • Fixed broken json.dumps using cloned instance. #46
    Source code(tar.gz)
    Source code(zip)
  • 0.22.2(Nov 30, 2020)

  • 0.22.1(Nov 27, 2020)

  • 0.22.0(Nov 27, 2020)

  • 0.21.1(Sep 30, 2020)

  • 0.21.0(Sep 22, 2020)

    • Added match utility method. #11 #16
    • Added indexes option support to keypaths method. #13
    • Updated keypaths method to use the default keypath_separator (.) instead of None.
    • Fixed keypath_separator inheritance when init from another benedict instance. #35
    • Fixed json.dumps no longer works directly with benedict. #34
    Source code(tar.gz)
    Source code(zip)
  • 0.20.0(Sep 20, 2020)

    • Added BaseDict as base class to keep pointer to the initial input dict. #32
    • Added automatic benedict casting to all methods that return dict instances.
    • Updated flatten method, now a KeyError is raised in case of existing key.
    Source code(tar.gz)
    Source code(zip)
  • 0.19.0(Sep 11, 2020)

    • Added plist format support.
    • Enforced IODict initial check when using filepath or data-string.
    • Improved KeyError messages. PR #28
    • Added encoding optional argument to io_util.read_file and io_util.write_file.
    • Fixed python 3.5/3.6 I/O encoding issue.
    Source code(tar.gz)
    Source code(zip)
  • 0.18.2(Sep 4, 2020)

  • 0.18.1(Aug 5, 2020)

    • Added data format auto-detection when creating instance with data from filepath or url.
    • Fixed keypath_separator support when using from_{format} methods.
    Source code(tar.gz)
    Source code(zip)
  • 0.18.0(Dec 21, 2021)

    • Added from_pickle and to_pickle methods.
    • Added PickleSerializer.
    • Added datetime, Decimal and set support to JSONSerializer.
    • Updated dump method to use JSONSerializer.
    • Refactored Base64Serializer.
    • Fixed type_util.is_json_serializable with set objects.
    • Fixed search method for int no results - #7
    • Improved invert method to handles correctly lists and tuples.
    • Improved io_util.read_file and io_util.write_file methods.
    • Improved code quality and CI.
    Source code(tar.gz)
    Source code(zip)
  • 0.17.0(Dec 21, 2021)

    • Added groupby utility method.
    • Added nest utility method.
    • Added keylists core method.
    • Reorganized lib and tests packages.
    • Improved code quality and CI.
    Source code(tar.gz)
    Source code(zip)
Owner
Fabio Caccamo
Python/Django, MySQL, JavaScript/jQuery/Vue.js, Node/Gulp/Sass, Objective-C, ...
Fabio Caccamo
This repository is a compilation of important Data Structures and Algorithms based on Python.

Python DSA ?? This repository is a compilation of important Data Structures and Algorithms based on Python. Please make seperate folders for different

Bhavya Verma 27 Oct 29, 2022
This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages

DSA-Code-Snippet This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages Cont

DSCSRMNCR 3 Oct 22, 2021
Data Structures and algorithms package implementation

Documentation Simple and Easy Package --This is package for enabling basic linear and non-linear data structures and algos-- Data Structures Array Sta

null 1 Oct 30, 2021
This repo is all about different data structures and algorithms..

Data Structure and Algorithm : Want to learn data strutrues and algorithms ??? Then Stop thinking more and start to learn today. This repo will help y

Priyanka Kothari 7 Jul 10, 2022
This repo represents all we learned and are learning in Data Structure course.

DataStructure Journey This repo represents all we learned and are learning in Data Structure course which is based on CLRS book and is being taught by

Aprime Afr (Alireza Afroozi) 3 Jan 22, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 1, 2023
A collection of data structures and algorithms I'm writing while learning

Data Structures and Algorithms: This is a collection of data structures and algorithms that I write while learning the subject Stack: stack.py A stack

Dhravya Shah 1 Jan 9, 2022
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Basic sort and search algorithms written in python.

Basic sort and search algorithms written in python. These were all developed as part of my Computer Science course to demonstrate understanding so they aren't 100% efficent

Ben Jones 0 Dec 14, 2022
IADS 2021-22 Algorithm and Data structure collection

A collection of algorithms and datastructures introduced during UoE's Introduction to Datastructures and Algorithms class.

Artemis Livingstone 20 Nov 7, 2022
CLASSIX is a fast and explainable clustering algorithm based on sorting

CLASSIX Fast and explainable clustering based on sorting CLASSIX is a fast and explainable clustering algorithm based on sorting. Here are a few highl

null 69 Jan 6, 2023
Tomli is a Python library for parsing TOML. Tomli is fully compatible with TOML v1.0.0.

Tomli A lil' TOML parser Table of Contents generated with mdformat-toc Intro Installation Usage Parse a TOML string Parse a TOML file Handle invalid T

Taneli Hukkinen 313 Dec 26, 2022
Get important strings inside [Info.plist] & and Binary file also all output of result it will be saved in [app_binary].json , [app_plist_file].json file

Get important strings inside [Info.plist] & and Binary file also all output of result it will be saved in [app_binary].json , [app_plist_file].json file

null 12 Sep 28, 2022
πŸ¦‰ Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

srsly: Modern high-performance serialization utilities for Python This package bundles some of the best Python serialization libraries into one standa

Explosion 329 Dec 28, 2022
Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark environment.

pyspark-anonymizer Python library which makes it possible to dynamically mask/anonymize data using JSON string or python dict rules in a PySpark envir

null 6 Jun 30, 2022
Fully Automated YouTube Channel ▢️with Added Extra Features.

Fully Automated Youtube Channel β–’β–ˆβ–€β–€β–ˆ β–ˆβ–€β–€β–ˆ β–€β–€β–ˆβ–€β–€ β–€β–€β–ˆβ–€β–€ β–ˆβ–‘β–‘β–ˆ β–ˆβ–€β–€β–„ β–ˆβ–€β–€ β–ˆβ–€β–€β–ˆ β–’β–ˆβ–€β–€β–„ β–ˆβ–‘β–‘β–ˆ β–‘β–‘β–ˆβ–‘β–‘ β–‘β–’β–ˆβ–‘β–‘ β–ˆβ–‘β–‘β–ˆ β–ˆβ–€β–€β–„ β–ˆβ–€β–€ β–ˆβ–„β–„β–€ β–’β–ˆβ–„β–„β–ˆ β–€β–€β–€β–€ β–‘β–‘β–€β–‘β–‘ β–‘β–’β–ˆβ–‘β–‘ β–‘β–€β–€β–€ β–€β–€β–€β–‘

sam-sepiol 249 Jan 2, 2023
A Web Scraper built with beautiful soup, that fetches udemy course information. Get udemy course information and convert it to json, csv or xml file

Udemy Scraper A Web Scraper built with beautiful soup, that fetches udemy course information. Installation Virtual Environment Firstly, it is recommen

Aditya Gupta 15 May 17, 2022
Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c.

Tablib: format-agnostic tabular dataset library _____ ______ ___________ ______ __ /_______ ____ /_ ___ /___(_)___ /_ _ __/_ __ `/__ _

Jazzband 4.2k Dec 30, 2022
Script to generate a massive volume of data in sql, csv, json or xml format

DataGenerator Made with Python Open for pull requests 1. Dependencies To install required dependencies run pip install -r requirements.txt 2. Executi

icrescenti 3 Sep 20, 2022
This Ivy plugin adds support for TOML file headers.

This Ivy plugin adds support for TOML file headers as an alternative to YAML.

Darren Mulholland 1 Nov 9, 2021