🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Overview

Boltons

boltons should be builtins.

Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously missing from — the standard library, including:

Full and extensive docs are available on Read The Docs. See what's new by checking the CHANGELOG.

Boltons is tested against Python 2.6, 2.7, 3.4, 3.5, 3.6, 3.7, 3.8 and 3.9, as well as CPython nightly and PyPy/PyPy3.

Installation

Boltons can be added to a project in a few ways. There's the obvious one:

pip install boltons

On macOS, it can also be installed via MacPorts:

sudo port install py-boltons

Then, thanks to PyPI, dozens of boltons are just an import away:

from boltons.cacheutils import LRU
my_cache = LRU()

However, due to the nature of utilities, application developers might want to consider other options, including vendorization of individual modules into a project. Boltons is pure-Python and has no dependencies. If the whole project is too big, each module is independent, and can be copied directly into a project. See the Integration section of the docs for more details.

Third-party packages

The majority of boltons strive to be "good enough" for a wide range of basic uses, leaving advanced use cases to Python's myriad specialized 3rd-party libraries. In many cases the respective boltons module will describe 3rd-party alternatives worth investigating when use cases outgrow boltons. If you've found a natural "next-step" library worth mentioning, see the next section!

Gaps

Found something missing in the standard library that should be in boltons? Found something missing in boltons? First, take a moment to read the very brief architecture statement to make sure the functionality would be a good fit.

Then, if you are very motivated, submit a Pull Request. Otherwise, submit a short feature request on the Issues page, and we will figure something out.

Comments
  • Add ioutils module

    Add ioutils module

    Spooled Temporary Files are file-like objects that start out mapped to in-memory objects, but automatically roll over to a temporary file once they reach a certain (configurable) threshhold. Unfortunately the built-in SpooledTemporaryFile class in Python does not implement the exact API that some common classes like StringIO do. SpooledTemporaryFile also spools all of it's in-memory files as cStringIO instances. cStringIO instances cannot be deep-copied, and they don't work with the zip library either. This along with the incompatible api makes it useless for several use-cases.

    To combat this but still gain the memory savings and usefulness of a true spooled file-like-object, two custom classes have been implemented which have a compatible API.

    SpooledBytesIO is a spooled file-like-object that only accepts bytes. On Python 2.x this means the 'str' type; on Python 3.x this means the 'bytes' type. Bytes are written in and retrieved exactly as given, but it will raise TypeErrors if something other than bytes are written.

    SpooledStringIO is a spooled file-like-object that only accepts unicode values. On Python 2.x this means the 'unicode' type and on Python 3.x this means the 'str' type. Values are accepted as unicode and then coerced into utf-8 encoded bytes for storage. On retrieval, the values are returned as unicode.

    opened by induane 20
  • Functions for time since UNIX epoch

    Functions for time since UNIX epoch

    I've always thought it was weird these were absent from the standard library, even though they're so simple. I added two functions to timeutils.py to transform between seconds since the UNIX epoch and datetime objects. Currently, they ignore any timezone information, but if you think these functions are a good fit for the package, I might be able to work on adding that kind of functionality.

    opened by tsupinie 18
  • Non-decorator version of funcutils.wraps and option to hide wrapped function.

    Non-decorator version of funcutils.wraps and option to hide wrapped function.

    Hi!

    As as discussed in #242, here is my suggestion to add an edge case to the usage of funcutils.wraps. The situation is that if the wrapped and wrapper function have the same arguments but differ as to which are keyword-only or positional-only, the function returned by wraps will fail as the wrong "invocation string" was inserted in the process. A solution to this is to create the internal FunctionBuilderinstance from the wrapper instead of the wrapped. As it didn't make sense to simply add an argument to wraps, I decided to replicate the structure of the built-in functools and add a update_wrapper function, of which wraps is only a partial call.

    This solves a problem that originated from using partials. Code that now works to wrap those is as follow:

    import functools
    from boltons import funcutils
    
    def func(a, b=1, c=1):  # Has the signature "a, b=1, c=1"
        return a, b, c
    
    wrapper = partial(func, b=2)
    functools.update_wrapper(wapper, func)  # Needed as FunctionBuilder will look for some attributes like __name__ that are not updated directly with `partial`.
    # wrapper has the signature; "a, *, b=2, c=1"
    
    new_func = funcutils.update_wrapper(wrapper, func, build_from=wrapper, injected='b')
    # new_func now has the signature : "a, *, c=1"
    new_func(1)  # Prints : 1, 2, 1, 
    

    Also, another small improvement I made here is to add an option to completely hide the wrapped function in the new one returned. Most IDEs use inspect.signature to extract the signature functions that they display to the user. However, by default, signature follows all wrapped functions until the innermost, so the user doesn't see the updated signature. One could use the follow_wrapped=False option, but I believe offering the possibility here could be nice.

    Closes #242

    opened by aulemahal 12
  • funcutils.wraps() with injected keyword does not work for keyword-only arguments.

    funcutils.wraps() with injected keyword does not work for keyword-only arguments.

    Given the following decorator:

    def inject_loop(func):
          sig = inspect.signature(func)
          loop_param = sig.parameters['loop'].replace(default=None)
          sig = sig.replace(parameters=[loop_param])
    
          def add_loop(
              args: typing.Tuple[typing.Any, ...],
              kwargs: typing.Dict[str, typing.Any]
          ) -> collections.OrderedDict:
              bargs = sig.bind(*args, **kwargs)
              bargs.apply_defaults()
              if bargs.arguments['loop'] is None:
                  bargs.arguments['loop'] = asyncio.get_event_loop()
    
              return bargs.arguments
    
          if inspect.isasyncgenfunction(func):
              async def async_gen_loop_wrapper(*args, **kwargs):
                  async for elem in func(**add_loop(args, kwargs)):
    >>                yield elem
              ret = async_gen_loop_wrapper
    
          elif inspect.iscoroutinefunction(func):
              async def async_loop_wrapper(*args, **kwargs):
                  return await func(**add_loop(args, kwargs))
              ret = async_loop_wrapper
    
          elif inspect.isgeneratorfunction(func):
              def gen_loop_wrapper(*args, **kwargs):
                  yield from func(**add_loop(args, kwargs))
              ret = gen_loop_wrapper
    
          else:
              def func_loop_wrapper(*args, **kwargs):
                  return func(**add_loop(args, kwargs))
              ret = func_loop_wrapper
    
          return boltons.funcutils.wraps(func, injected=['loop'])(ret)
    

    The following code fails:

    @inject_loop
    def example(*, loop):  # loop is a keyword-only argument
        return loop
    

    with the following stacktrace:

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    .../python3.6/site-packages/boltons/funcutils.py in remove_arg(self, arg_name)
        565         try:
    --> 566             self.args.remove(arg_name)
        567         except ValueError:
    
    ValueError: list.remove(x): x not in list
    
    During handling of the above exception, another exception occurred:
    
    MissingArgument                           Traceback (most recent call last)
    <ipython-input-3-6ef3f4f5dc19> in <module>()
    ----> 1 @lib.inject_loop
          2 def example(*, loop):
          3     return loop
    
    ... in inject_loop(func)
        169         ret = func_loop_wrapper
        170
    --> 171     return boltons.funcutils.wraps(func, injected=['loop'])(ret)
        172
        173
    
    .../python3.6/site-packages/boltons/funcutils.py in wraps(func, injected, **kw)
        295     for arg in injected:
        296         try:
    --> 297             fb.remove_arg(arg)
        298         except MissingArgument:
        299             if inject_to_varkw and fb.varkw is not None:
    
    .../python3.6/site-packages/boltons/funcutils.py in remove_arg(self, arg_name)
        569                                   % (arg_name, self.name, self.args))
        570             exc.arg_name = arg_name
    --> 571             raise exc
        572         d_dict.pop(arg_name, None)
        573         self.defaults = tuple([d_dict[a] for a in self.args if a in d_dict])
    
    MissingArgument: arg 'loop' not found in example argument list: []
    

    However, it works if I do this:

    @inject_loop
    def example(loop):  # Not a keyword-only argument
        return loop
    

    I believe the solution should also check the kwonly args when attempting to mark a parameter as "injected", so that keyword only arguments can be listed in the injected parameter to wraps().

    opened by xlorepdarkhelm 12
  • Add FrozenDict type to dictutils

    Add FrozenDict type to dictutils

    This is another tool I have built a number of times in various permutations; I don't know if it's all that interesting but I've found it quite useful in a number of situations including immutable constant maps without heinous amounts of namedtuple boilerplate, and some cases with threading.

    A FrozenDict is a dictionary whose values are set during instance creation and are fixed from that point on. Setting and altering values is not allowed. This can be useful in a number of scenarios, including setting up mapping constants without worrying that the values will get mutated by a misbehaving function.

    There are a lot of online recipes for creating a FrozenDict class, but most still rely on a real dictionary under the hood for storage. They also tend to be extra weighty because they have an underlying object dict.

    One common solution is to use a named tuple, but this requires setting up boilerplate for every type or relying on factory functions. The FrozenDict utilizes a named tuple for storage and then is further made lighter by utilizing slots to eliminate the underlying object dict.

    opened by induane 12
  • Why IndexedSet not update the index of items?

    Why IndexedSet not update the index of items?

    What is the motivation of not updating the current index of items when they are removed? It's this a bug? Because i starting using IndexedSet to pop items from a list that i have and i start get index of range errors.

    https://github.com/mahmoud/boltons/blob/bf8a65942cd8078dba7dc45543ae0923fe2fabbc/boltons/setutils.py#L201-L209

    opened by Urahara 11
  • PermissionError with AtomicSaver on Windows

    PermissionError with AtomicSaver on Windows

    If I try this little code on Windows 7:

    from boltons.fileutils import AtomicSaver
    with AtomicSaver('foo.txt') as f:
        f.write('whatever')
    

    I get the following Exception:

    Traceback (most recent call last):
      File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 272, in setup
        overwrite=self.overwrite_part)
      File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 194, in _atomic_rename
        os.rename(path, new_path)
    PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem ande
    ren Prozess verwendet wird: 'C:\\Windows\\System32\\tmphia0tzzm' -> 'foo.txt.part'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 281, in __enter__
        self.setup()
      File "C:\Python34\lib\site-packages\boltons\fileutils.py", line 274, in setup
        os.unlink(tmp_part_path)
    PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem ande
    ren Prozess verwendet wird: 'C:\\Windows\\System32\\tmphia0tzzm'
    
    opened by stlehmann 11
  • add new file procutils, for process-related utilities

    add new file procutils, for process-related utilities

    Add a more flexible interface to subprocess, in the style of subprocess.call() but better.

    I find subprocess.call(), check_call() and check_output() really badly lacking, but I end up using them anyway due to the considerable extra complexity in using my own Popen object.

    For example:

    • check_* does not report an exit code or stderr in the exception, so you may know "something failed" but it's hard to tell what or why.
    • *_call use the process's stdout, resulting in often unwanted extra output
    • with any of these calls, it is impossible to interact with stdin

    I have endeavoured to create a function that is very flexible but with excellent defaults, in the style of requests.get(), where the 90% use case is very simple, but the 99% use case is still possible. In getting into specifics (such as having None redirect output to /dev/null), I'm pretty sure I broke windows support. It's probably possible to add with minor changes in interface though.

    opened by ekimekim 10
  • strip() that works with iterables

    strip() that works with iterables

    I'd like strip / rstrip / lstrip that works with arbitrary iterables:

    def iter_lstrip(li, it): 
        pos = 0
        while li[pos] == it:
            pos += 1
            if pos == len(li):
                return []
        return li[pos:]
    
    
    def iter_rstrip(li, it):
        return list(reversed(iter_lstrip(list(reversed(li)), it)))
    
    
    def iter_strip(li, it):
        return iter_lstrip(iter_rstrip(li, it), it)
    
    iter_strip([0,0,0,1,2,3,4,0,0], 0)
    # [1,2,3,4]
    
    opened by kurtbrose 9
  • Fix issues with LRI Cache #155

    Fix issues with LRI Cache #155

    Adds failing tests to exercise issues with LRI

    To fix the issue, I have a trivial solution that has LRI inherit from LRU. After a couple hours of thinking about it I don't think there's a faster solution that using the linked list that LRU uses without introducing an unbounded memory size of the underlying datastructure stored.

    That being said LRU has a bunch of thread locking logic that I'm uncertain you would want to add to the LRI cache. So the current solution is more of a conversation piece. Open to feedback on how to go forward on this:

    1. Extract common logic from LRU and have LRI and LRU utilize that? (lets us omit thread safety from LRI if we want)
    2. Keep naive solution (though thread locking might introduce unwanted overhead).
    3. Maybe make thread safety an optional feature on both LRU and LRI. Would be tricky not to break existing features. Probably would require names like: LRU, ThreadUnsafeLRU, LRI, ThreadSafeLRI.
    opened by CameronCairns 8
  • complement set

    complement set

    the idea is complements of sets -- like inverse sets -- they support all of the set API with each other and with regular sets with the exception of iteration and len (b/c they contain "everything")

    I really badly wanted one of these to be able to pass in a "universal set" as a null value rather than None -- would have saved a bunch of code like

    for item in things:
        if filter is not None:
            if item not in filter:
                continue
    

    instead, complement(set()) would simply return True for everything (or everything that is hashable)

    the next day a co-worker wanted one of these for a different purpose, so I took a couple hours to bang out some code and the implementation actually seems pretty straight forward

    still need to implement the operator overloads, more docs and better tests

    opening the PR early to allow for feedback ASAP

    opened by kurtbrose 8
  • Function to format a list with commas and a final 'and'/'or'

    Function to format a list with commas and a final 'and'/'or'

    A function that I frequently want is to print a list of strings, separating them with commas, plus a final conjunction like 'and' or 'or'. The Oxford comma should be the default but some users might like to disable it.

    >>> human_list(['spam'])
    'spam'
    >>> human_list(['egg', 'spam'])
    'egg and spam'
    >>> human_list(['egg', 'bacon', 'spam'], conjunction='or')
    'egg, bacon, or spam'
    >>> human_list(['spam', 'spam', 'egg', 'bacon', 'spam'])
    'spam, spam, egg, bacon, and spam'
    >>> human_list(['lobster thermidor aux crevettes', 'garnished with truffle paté', 'with a fried egg on top', 'spam'], oxford=False)
    'lobster thermidor aux crevettes, garnished with truffle paté, with a fried egg on top and spam'
    
    opened by lordmauve 0
  • cookbooks code examples are all one liners

    cookbooks code examples are all one liners

    Hi would you please fix your remap cookbook (https://sedimental.org/remap.html), it looks like this would be so amazingly useful but all the code examples are showing up as one liners.

    image

    Thank you in advance.

    opened by jennifer-klemisch-seagen 0
  • Support latest Python versions

    Support latest Python versions

    I noticed README says:

    Boltons is tested against [...] CPython nightly

    But I cannot find where this test occurs. There seem to be no 3.10 test runs or CPython nightly test runs. My concern is the traceback string format has changed in 3.11 so the parser may not work as is.

    opened by cretz 0
  • add chunk_ranges function to iterutils

    add chunk_ranges function to iterutils

    This PR adds a chunk_ranges function to iterutils, which doesn't chunk an iterable or list directly, but produces the relevant indices to chunk it. It contains functionality for windowing/overlap (see also issue https://github.com/mahmoud/boltons/issues/310) and alignment of the chunks. I'm not sure if input_size or input_end is more appropriate for the arguments, also the function name might need improvement.

    It would be great to incorporate this function into boltons IMO, thanks a lot for considering this.

    This PR includes the function itself, a test, and updates the docs.

    opened by jstriebel 1
Releases(21.0.0)
Owner
Mahmoud Hashemi
Code goes here, but also @hatnote & @SimpleLegal. @paypal alumnus. Belaboring the finer points of software. glomming, but mostly 2020ing atm.
Mahmoud Hashemi
Runes - Simple Cookies You Can Extend (similar to Macaroons)

Runes - Simple Cookies You Can Extend (similar to Macaroons) is a paper called "Macaroons: Cookies with Context

Rusty Russell 22 Dec 11, 2022
A functional standard library for Python.

Toolz A set of utility functions for iterators, functions, and dictionaries. See the PyToolz documentation at https://toolz.readthedocs.io LICENSE New

null 4.1k Dec 30, 2022
Standard implementations of FedLab and its provided benchmarks.

FedLab-benchmarks This repo contains standard implementations of FedLab and its provided benchmarks. Currently, following algorithms or benchrmarks ar

SMILELab-FL 104 Dec 5, 2022
Cleaning-utils - a collection of small Python functions and classes which make cleaning pipelines shorter and easier

cleaning-utils [] [] [] cleaning-utils is a collection of small Python functions

null 4 Aug 31, 2022
SysInfo is an app developed in python which gives Basic System Info , and some detailed graphs of system performance .

SysInfo SysInfo is an app developed in python which gives Basic System Info , and some detailed graphs of system performance . Installation Download t

null 5 Nov 8, 2021
A python app which aggregates and splits costs from multiple public cloud providers into a csv

Cloud Billing This project aggregates the costs public cloud resources by accounts, services and tags by importing the invoices from public cloud prov

null 1 Oct 4, 2022
A utility that makes it easy to work with Python projects containing lots of packages, of which you only want to develop some.

Mixed development source packages on top of stable constraints using pip mxdev [mɪks dɛv] is a utility that makes it easy to work with Python projects

BlueDynamics Alliance 6 Jun 8, 2022
A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Ubelt is a small library of robust, tested, documented, and simple functions that extend the Python standard library. It has a flat API that all behav

Jon Crall 638 Dec 13, 2022
Manage your exceptions in Python like a PRO

A linter to manage all your python exceptions and try/except blocks (limited only for those who like dinosaurs).

Guilherme Latrova 353 Dec 31, 2022
Astvuln is a simple AST scanner which recursively scans a directory, parses each file as AST and runs specified method.

Astvuln Astvuln is a simple AST scanner which recursively scans a directory, parses each file as AST and runs specified method. Some search methods ar

Bitstamp Security 7 May 29, 2022
A clock app, which helps you with routine tasks.

Clock This app helps you with routine tasks. Alarm Clock Timer Stop Watch World Time (Which city you want) About me Full name: Matin Ardestani Age: 14

Matin Ardestani 13 Jul 30, 2022
one_click_kag_server is a program which tries to fully automate the creation of a King Arthur's Gold server.

one_click_kag_server is a program which tries to fully automate the creation of a King Arthur's Gold server.

Benjamin Gorman 4 Jan 5, 2022
New time-based UUID formats which are suited for use as a database key

uuid6 New time-based UUID formats which are suited for use as a database key. This module extends immutable UUID objects (the UUID class) with the fun

null 26 Dec 30, 2022
UUID version 7, which are time-sortable (following the Peabody RFC4122 draft)

uuid7 - time-sortable UUIDs This module implements the version 7 UUIDs, proposed by Peabody and Davis in https://www.ietf.org/id/draft-peabody-dispatc

Steve Simmons 22 Dec 20, 2022
glip is a module for retrieve ip address like local-ip, global-ip, external-ip as string.

gle_ip_info glip is a module for retrieve ip address like local-ip, global-ip, external-ip as string.

Fatin Shadab 3 Nov 21, 2021
ticktock is a minimalist library to view Python time performance of Python code.

ticktock is a minimalist library to view Python time performance of Python code.

Victor Benichoux 30 Sep 28, 2022
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various editors to quickly sort all your imports.

Python Code Quality Authority 5.5k Jan 8, 2023
A Python library for reading, writing and visualizing the OMEGA Format

A Python library for reading, writing and visualizing the OMEGA Format, targeted towards storing reference and perception data in the automotive context on an object list basis with a focus on an urban use case.

Institut für Kraftfahrzeuge, RWTH Aachen, ika 12 Sep 1, 2022
RapidFuzz is a fast string matching library for Python and C++

RapidFuzz is a fast string matching library for Python and C++, which is using the string similarity calculations from FuzzyWuzzy

Max Bachmann 1.7k Jan 4, 2023