A fancy and practical functional tools

Overview

Funcy Build Status Join the chat at https://gitter.im/Suor/funcy

A collection of fancy functional tools focused on practicality.

Inspired by clojure, underscore and my own abstractions. Keep reading to get an overview or read the docs. Or jump directly to cheatsheet.

Works with Python 2.7, 3.4+ and pypy.

Installation

pip install funcy

Overview

Import stuff from funcy to make things happen:

from funcy import whatever, you, need

Merge collections of same type (works for dicts, sets, lists, tuples, iterators and even strings):

merge(coll1, coll2, coll3, ...)
join(colls)
merge_with(sum, dict1, dict2, ...)

Walk through collection, creating its transform (like map but preserves type):

walk(str.upper, {'a', 'b'})            # {'A', 'B'}
walk(reversed, {'a': 1, 'b': 2})       # {1: 'a', 2: 'b'}
walk_keys(double, {'a': 1, 'b': 2})    # {'aa': 1, 'bb': 2}
walk_values(inc, {'a': 1, 'b': 2})     # {'a': 2, 'b': 3}

Select a part of collection:

select(even, {1,2,3,10,20})                  # {2,10,20}
select(r'^a', ('a','b','ab','ba'))           # ('a','ab')
select_keys(callable, {str: '', None: None}) # {str: ''}
compact({2, None, 1, 0})                     # {1,2}

Manipulate sequences:

take(4, iterate(double, 1)) # [1, 2, 4, 8]
first(drop(3, count(10)))   # 13

lremove(even, [1, 2, 3])    # [1, 3]
lconcat([1, 2], [5, 6])     # [1, 2, 5, 6]
lcat(map(range, range(4)))  # [0, 0, 1, 0, 1, 2]
lmapcat(range, range(4))    # same
flatten(nested_structure)   # flat iter
distinct('abacbdd')         # iter('abcd')

lsplit(odd, range(5))       # ([1, 3], [0, 2, 4])
lsplit_at(2, range(5))      # ([0, 1], [2, 3, 4])
group_by(mod3, range(5))    # {0: [0, 3], 1: [1, 4], 2: [2]}

lpartition(2, range(5))     # [[0, 1], [2, 3]]
chunks(2, range(5))         # iter: [0, 1], [2, 3], [4]
pairwise(range(5))          # iter: [0, 1], [1, 2], ...

And functions:

partial(add, 1)             # inc
curry(add)(1)(2)            # 3
compose(inc, double)(10)    # 21
complement(even)            # odd
all_fn(isa(int), even)      # is_even_int

one_third = rpartial(operator.div, 3.0)
has_suffix = rcurry(str.endswith)

Create decorators easily:

@decorator
def log(call):
    print call._func.__name__, call._args
    return call()

Abstract control flow:

walk_values(silent(int), {'a': '1', 'b': 'no'})
# => {'a': 1, 'b': None}

@once
def initialize():
    "..."

with suppress(OSError):
    os.remove('some.file')

@ignore(ErrorRateExceeded)
@limit_error_rate(fails=5, timeout=60)
@retry(tries=2, errors=(HttpError, ServiceDown))
def some_unreliable_action(...):
    "..."

class MyUser(AbstractBaseUser):
    @cached_property
    def public_phones(self):
        return self.phones.filter(public=True)

Ease debugging:

squares = {tap(x, 'x'): tap(x * x, 'x^2') for x in [3, 4]}
# x: 3
# x^2: 9
# ...

@print_exits
def some_func(...):
    "..."

@log_calls(log.info, errors=False)
@log_errors(log.exception)
def some_suspicious_function(...):
    "..."

with print_durations('Creating models'):
    Model.objects.create(...)
    # ...
# 10.2 ms in Creating models

And much more.

Dive in

Funcy is an embodiment of ideas I explain in several essays:

Running tests

To run the tests using your default python:

pip install -r test_requirements.txt
py.test

To fully run tox you need all the supported pythons to be installed. These are 2.6+, 3.3+, PyPy and PyPy3. You can run it for particular environment even in absense of all of the above:

tox -e py27
tox -e py36
tox -e lint
Comments
  • Argument collision with @decorator

    Argument collision with @decorator

    Logically obvious limitation without an obviously documented solution.

    Suppose a function decorated by a decorator created with @funcy.decorator actually has arguments named _args, _kwargs, or _func (it shouldn't, but supposing it did).

    Reasonably and correctly, call._args, call._kwargs, and call._func still resolve as expected and documented - good so far.

    An intuitive solution if you did want to inspect those arguments would be to call @decorator's __getattr__(self, ...) manually.

    But that unconditionally sets self.__dict__[...], so for example doing call.__getattr__('_args') can break call(), because call._args gets replaced.

    So the only safe way to access those arguments is to manually use the undocumented funcy.decorator.arggetter directly, or to manually implement the same logic of looking up the argument from call._args and call._kwargs based on the signature gotten from call._func.

    Actually, I think this problem might possibly effect all of the dunder methods too.

    Anyway, I think this could be solved by either:

    1. Just officially saying that as a special case, accessing those arguments to the wrapped function using the same built-in logic for getting arguments is completely unsupported.

    2. Officially documenting funcy.decorator.arggetter as the way to get those arguments.

    3. Using self.__dict__.setdefault instead of self.__dict__[...] in __getattr__, and documenting manually calling call.__getattr__ as the way.

    4. Exposing some sort of new method on the Call class like call._getarg, which could use either the 2nd or 3rd option internally.

    Thoughts?

    opened by mentalisttraceur 19
  • Pipe, thread

    Pipe, thread

    Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

    opened by berrytj 16
  • Feature request: Support method chaining

    Feature request: Support method chaining

    I would find method chaining very useful. This style is made available in many similar libraries, e.g. underscore, lodash, pydash and pandas.

    This is the only feature I've really been missing in funcy.

    opened by taleinat 14
  • Play nicely with static code analyzers

    Play nicely with static code analyzers

    In my case, PyCharm has trouble understanding things coming from funcy. Specifically, it fails to understand importing directly from the funcy package rather than the specific sub-modules, i.e. from funcy import merge causes it to mark the import as problematic. I imagine it has other issues too, I haven't investigated further (yet).

    I can think of several ways to improve the situation:

    1. Re-work the fancy importing mechanisms so that static analyzers do a better job of understanding them.
    2. Include stubs which will directly inform static analyzers about the structure of the funcy library and its contents
    3. Contribute stubs for funcy to the typeshed project, which is used by various static analyzers and IDEs.

    I'd be happy to help with this, if the maintainer(s) are open to it.

    opened by taleinat 14
  • All functions in the library should be curried.

    All functions in the library should be curried.

    I think making all functions from funcy automatically curried would simplify a vast number of use cases. For example, for an extract_int example (where extract_int("Hello, 123! We are 456!") => [123, 456]), without autocurry, the current best way to do this is:

    from funcy import map, partial, rcompose, re_all
    extract_int = rcompose(partial(re_all, r'\d+'), partial(map, int))
    

    Of course, we could use autocurry on them ourselves - but this doesn't seem to work well with some functions (I tested only with map and re_all, surprisingly re_all works):

    >>> from funcy import map, autocurry
    >>> map = autocurry(map)
    >>> map(int)([1, 2, 3])
    # .... blah error stuff ....
    TypeError: map() requires at least two args
    

    This and other such problems could be averted by having all functions (at least those in the library) automatically curried.

    looking for more interest 
    opened by ghost 14
  • Allow specifying an error handler for memoize

    Allow specifying an error handler for memoize

    We ran into an issue where we'd still like an exception to be memoized and return the exception when using the memoize decorator. A way to specify an error handler function would be appreciated. Or provide a list of exception types which should be cached.

    opened by richard-engineering 11
  • Adding a `flip` function to funcy

    Adding a `flip` function to funcy

    Hi,

    every time I try to code in a more functional manner in Python I find myself writing this little function sooner or later:

    
    flip = lambda f: lambda x,y: f(y,x)
    

    it's especially useful with currying, which is already provided by the library, so I think it would be a good addition. A very simple example:

    
    has_suffix = curry(flip(str.endswith))
    filter(has_suffix("a"), ["ab", "ba", "cd"])
    

    One thing to think about is the semantics of flip when used on a function with more than two arguments and on functions with **kwargs, especially the former case is interesting, as it could mean:

    
    flip = lambda f: lambda a,b,*args: f(b,a,*args)
    

    or

    
    flip = lambda f: lambda *args: f(*(args[1:] + [args[0]]))
    

    The latter meaning would be consistent with for example rotatef from CL, but on the other hand it could be less practical.

    opened by piotrklibert 11
  • Add kwargs support for memoize

    Add kwargs support for memoize

    Happy to update docs

    TODO: check performance impact of using inspect.getcallargs (it on its own does fallbacks depending how signature looks like)

    @Suor should I add feature proposal Issue on GH ?

    opened by lukmdo 11
  • `join_with` - transform single elements too

    `join_with` - transform single elements too

    Hi! funcy is a great tool. I especially like join_with because it easily and fast merges many dicts into one in, for example, a defaultdict(list) style. However, I noticed that if, for example, a list of dicts contains only one element - join_with doesn't change its structure. It could be very useful to change them too:

    list_of_dicts = [{1: 2, 3: 4, "foo": "bar"}]
    
    # current implementation
    join_with(list, list_of_dicts) -> {1: 2, 3: 4, "foo": "bar"}
    
    # proposed implementation
    join_with(list, list_of_dicts) -> {1: [2], 3: [4], "foo": ["bar"]}
    
    opened by meh-wzdech 10
  • Join the forces

    Join the forces

    Python has atleast 3 great functional programming libraries. These libraries have lots of overlapping functionality:

    • https://github.com/pytoolz/toolz
    • https://github.com/Suor/funcy
    • https://github.com/kachayev/fn.py

    Maybe also this:

    • https://github.com/Suor/whatever

    I would see as the benefit of the whole community if the authors of all these libraries could work more together on a common codebase (I'm willing to participate also in making this happen).

    @Suor, @pytoolz, @mrocklin, @kachayev Thank you for the hard work all of you have put into these wonderful packages! What do you guys think about this? Could we make more collaboration between the projects?

    opened by kvesteri 10
  • Make get_in, set_in work with lists/indexables

    Make get_in, set_in work with lists/indexables

    Currently:

    def get_in(coll, path, default=None):
        for key in path:
            if key in coll:
                coll = coll[key]
            else:
                return default
        return coll
    

    With current implementation, get_in([1, 2, 3, 4], [0]) returns None, while get_in([1,2,3,4], [1]) returns 2, due to the in behavior, which is slightly funky.

    I've modified get_in and set_in to work with more general indexables - set_in only with lists but get_in with general indexables.

    If this change sounds good, happy to add to docs as well.

    Thanks!

    opened by mcamac 9
  • Unprecise documentation on extended function semantics

    Unprecise documentation on extended function semantics

    I stumbled mentions on extended function semantics and went to this section of the doc: https://funcy.readthedocs.io/en/stable/extended_fns.html#extended-fns

    IMHO, this section could use some examples. For example it states that providing a mapping instead of a callable would result in lambda x: f[x] being used. It took me some time to figure out how to use this just by reading this dry description.

    Would you be interested that I complete this page a bit with usage examples?

    opened by antoine-gallix 1
  • feat($retry): enable capped exponential backoff and jitter via keyword arguments

    feat($retry): enable capped exponential backoff and jitter via keyword arguments

    Summary

    • Enable exponential backoff via keyword arguments.

    Checklist

    • [x] Implement unit test cases.

    Reason

    • Currently, the feature supports highly customized retry intervals via a pass-in callable argument.
    • However, it's hard to comprehend that the argument timeout can also accept a callable object without tracing the source code.
    • Capped exponential backoffs and jitter are already well-known approaches, maybe you can consider making them specific features via keyword arguments.
    opened by RainrainWu 0
  • funcy.autocurry doesn't work with functions with key-words only

    funcy.autocurry doesn't work with functions with key-words only

    Hi! I'm happy to use the library but looks like I found a bug:

    >>>@funcy.autocurry
    ...def a(b, c): return b + c
    
    >>>a(c=1)(2)
    3
    
    >>>@funcy.autocurry
    ...def aa(b, *, c): return b + c
    
    >>>aa(c=1)
    Traceback (most recent call last):
       ...
    TypeError: aa() missing 1 required keyword-only argument: 'c'
    

    funcy version: 1.17

    opened by ADR-007 2
  • rcurry and similar functions do not work with methods of built-in types

    rcurry and similar functions do not work with methods of built-in types

    >>> has_suffix = rcurry(str.endswith)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\funcs.py", line 57, in rcurry
        n = get_spec(func).max_n
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\_inspect.py", line 108, in get_spec
        mod = func.__module__
    AttributeError: 'method_descriptor' object has no attribute '__module__'. Did you mean: '__reduce__'?```
    looking for more interest 
    opened by NullSurplus 6
  • Add type stubs

    Add type stubs

    Stub files can be added to the project to provide static type information that can be used by tools like mypy.

    I believe that the simplest way to add this functionality to funcy is to provide stub files along with their respective modules. That is, for example, a stub file colls.pyi corresponding to the contents of colls.py; a funcs.pyi corresponding to funcs.py and so on. Corresponding snippets from funcs.py and funcs.pyi would look like:

    funcs.py

    def identity(x):
        return x
    
    def constantly(x):
        return lambda *a, **kw: x
    
    def caller(*a, **kw):
        return lambda f: f(*a, **kw)
    

    funcs.pyi

    from typing import Any, Callable, TypeVar
    
    T = TypeVar('T')
    
    def identity(x: T) -> T: ...
    def constantly(x: T) -> Callable[..., T]: ...
    def caller(*a: Any, **kw: Any) -> Callable[[Callable[..., T]], T]: ...
    

    Take a look at more-itertools to see how this looks like when fully implemented.

    If you are interested in this functionality, I could perhaps submit a draft PR by the end of the week.

    opened by ruancomelli 18
Owner
Alexander Schepanovski
Alexander Schepanovski
Functional UUIDs for Python.

??️FUUID stands for Functional Universally Unique IDentifier. FUUIDs are compatible with regular UUIDs but are naturally ordered by generation time, collision-free and support succinct representations such as raw binary and base58-encoded strings.

Phil Demetriou 147 Oct 27, 2022
A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Ubelt is a small library of robust, tested, documented, and simple functions that extend the Python standard library. It has a flat API that all behav

Jon Crall 638 Dec 13, 2022
Tools to connect to and interact with the Mila cluster

milatools The milatools package provides the mila command, which is meant to help with connecting to and interacting with the Mila cluster. Install Re

Mila 32 Dec 1, 2022
A collection of resources/tools and analyses for the angr binary analysis framework.

Awesome angr A collection of resources/tools and analyses for the angr binary analysis framework. This page does not only collect links and external r

null 105 Jan 2, 2023
Helper script to bootstrap a Python environment with the tools required to build and install packages.

python-bootstrap Helper script to bootstrap a Python environment with the tools required to build and install packages. Usage $ python -m bootstrap.bu

Filipe Laíns 7 Oct 6, 2022
Obsidian tools - a Python package for analysing an Obsidian.md vault

obsidiantools is a Python package for getting structured metadata about your Obsidian.md notes and analysing your vault.

Mark Farragher 153 Jan 4, 2023
'ToolBurnt' A Set Of Tools In One Place =}

'ToolBurnt' A Set Of Tools In One Place =}

MasterBurnt 5 Sep 10, 2022
PyGMT - A Python interface for the Generic Mapping Tools

PyGMT A Python interface for the Generic Mapping Tools Documentation (development version) | Contact | Try Online Why PyGMT? A beautiful map is worth

The Generic Mapping Tools (GMT) 564 Dec 28, 2022
This is Cool Utility tools that you can use in python.

This is Cool Utility tools that you can use in python. There are a few tools that you might find very useful, you can use this on pretty much any project and some utils might help you a lot and save so much time since it’s a simple function.

Senarc Studios 6 Apr 18, 2022
Tools for binary data on cassette

Micro Manchester Tape Storage Tools for storing binary data on cassette Includes: Python script for encoding Arduino sketch for decoding Eagle CAD fil

Zack Nelson 28 Dec 25, 2022
Multipurpose Growtopia Server tools, can be used for newbie to learn things.

Information Multipurpose Growtopia Server tools, can be used for newbie to learn things. Requirements - Python 3.x - Operating System (Recommended : W

Morphias 2 Oct 29, 2021
A collection of tools for biomedical research assay analysis in Python.

waltlabtools A collection of tools for biomedical research assay analysis in Python. Key Features Analysis for assays such as digital ELISA, including

Tyler Dougan 1 Apr 18, 2022
A primitive Python wrapper around the Gromacs tools.

README: GromacsWrapper A primitive Python wrapper around the Gromacs tools. The library is tested with GROMACS 4.6.5, 2018.x, 2019.x, 2020.x, and 2021

Becksteinlab 140 Dec 28, 2022
EVE-NG tools, A Utility to make operations with EVE-NG more friendly.

EVE-NG tools, A Utility to make operations with EVE-NG more friendly. Also it support different snapshot operations with same style as Libvirt/KVM

Bassem Aly 8 Jan 5, 2023
This utility synchronises spelling dictionaries from various tools with each other.

This utility synchronises spelling dictionaries from various tools with each other. This way the words that have been trained on MS Office are also correctly checked in vim or Firefox. And vice versa of course.

Patrice Neff 2 Feb 11, 2022
This is discord nitro code generator and checker made with python. This will generate nitro codes and checks if the code is valid or not. If code is valid then it will print the code leaving 2 lines and if not then it will print '*'.

Discord Nitro Generator And Checker ⚙️ Rᴜɴ Oɴ Rᴇᴘʟɪᴛ ??️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs If you are taking code from this repository without a fork, then atleast

Vɪɴᴀʏᴀᴋ Pᴀɴᴅᴇʏ 37 Jan 7, 2023
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

Mahmoud Hashemi 6k Jan 4, 2023
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various editors to quickly sort all your imports.

Python Code Quality Authority 5.5k Jan 8, 2023
A library for interacting with Path of Exile game and economy data, and a unique loot filter generation framework.

wraeblast A library for interfacing with Path of Exile game and economy data, and a set of item filters geared towards trade league players. Filter Ge

David Gidwani 29 Aug 28, 2022