A fancy and practical functional tools

Overview

Funcy Build Status Join the chat at https://gitter.im/Suor/funcy

A collection of fancy functional tools focused on practicality.

Inspired by clojure, underscore and my own abstractions. Keep reading to get an overview or read the docs. Or jump directly to cheatsheet.

Works with Python 2.7, 3.4+ and pypy.

Installation

pip install funcy

Overview

Import stuff from funcy to make things happen:

from funcy import whatever, you, need

Merge collections of same type (works for dicts, sets, lists, tuples, iterators and even strings):

merge(coll1, coll2, coll3, ...)
join(colls)
merge_with(sum, dict1, dict2, ...)

Walk through collection, creating its transform (like map but preserves type):

walk(str.upper, {'a', 'b'})            # {'A', 'B'}
walk(reversed, {'a': 1, 'b': 2})       # {1: 'a', 2: 'b'}
walk_keys(double, {'a': 1, 'b': 2})    # {'aa': 1, 'bb': 2}
walk_values(inc, {'a': 1, 'b': 2})     # {'a': 2, 'b': 3}

Select a part of collection:

select(even, {1,2,3,10,20})                  # {2,10,20}
select(r'^a', ('a','b','ab','ba'))           # ('a','ab')
select_keys(callable, {str: '', None: None}) # {str: ''}
compact({2, None, 1, 0})                     # {1,2}

Manipulate sequences:

take(4, iterate(double, 1)) # [1, 2, 4, 8]
first(drop(3, count(10)))   # 13

lremove(even, [1, 2, 3])    # [1, 3]
lconcat([1, 2], [5, 6])     # [1, 2, 5, 6]
lcat(map(range, range(4)))  # [0, 0, 1, 0, 1, 2]
lmapcat(range, range(4))    # same
flatten(nested_structure)   # flat iter
distinct('abacbdd')         # iter('abcd')

lsplit(odd, range(5))       # ([1, 3], [0, 2, 4])
lsplit_at(2, range(5))      # ([0, 1], [2, 3, 4])
group_by(mod3, range(5))    # {0: [0, 3], 1: [1, 4], 2: [2]}

lpartition(2, range(5))     # [[0, 1], [2, 3]]
chunks(2, range(5))         # iter: [0, 1], [2, 3], [4]
pairwise(range(5))          # iter: [0, 1], [1, 2], ...

And functions:

partial(add, 1)             # inc
curry(add)(1)(2)            # 3
compose(inc, double)(10)    # 21
complement(even)            # odd
all_fn(isa(int), even)      # is_even_int

one_third = rpartial(operator.div, 3.0)
has_suffix = rcurry(str.endswith)

Create decorators easily:

@decorator
def log(call):
    print call._func.__name__, call._args
    return call()

Abstract control flow:

walk_values(silent(int), {'a': '1', 'b': 'no'})
# => {'a': 1, 'b': None}

@once
def initialize():
    "..."

with suppress(OSError):
    os.remove('some.file')

@ignore(ErrorRateExceeded)
@limit_error_rate(fails=5, timeout=60)
@retry(tries=2, errors=(HttpError, ServiceDown))
def some_unreliable_action(...):
    "..."

class MyUser(AbstractBaseUser):
    @cached_property
    def public_phones(self):
        return self.phones.filter(public=True)

Ease debugging:

squares = {tap(x, 'x'): tap(x * x, 'x^2') for x in [3, 4]}
# x: 3
# x^2: 9
# ...

@print_exits
def some_func(...):
    "..."

@log_calls(log.info, errors=False)
@log_errors(log.exception)
def some_suspicious_function(...):
    "..."

with print_durations('Creating models'):
    Model.objects.create(...)
    # ...
# 10.2 ms in Creating models

And much more.

Dive in

Funcy is an embodiment of ideas I explain in several essays:

Running tests

To run the tests using your default python:

pip install -r test_requirements.txt
py.test

To fully run tox you need all the supported pythons to be installed. These are 2.6+, 3.3+, PyPy and PyPy3. You can run it for particular environment even in absense of all of the above:

tox -e py27
tox -e py36
tox -e lint
Comments
  • Argument collision with @decorator

    Argument collision with @decorator

    Logically obvious limitation without an obviously documented solution.

    Suppose a function decorated by a decorator created with @funcy.decorator actually has arguments named _args, _kwargs, or _func (it shouldn't, but supposing it did).

    Reasonably and correctly, call._args, call._kwargs, and call._func still resolve as expected and documented - good so far.

    An intuitive solution if you did want to inspect those arguments would be to call @decorator's __getattr__(self, ...) manually.

    But that unconditionally sets self.__dict__[...], so for example doing call.__getattr__('_args') can break call(), because call._args gets replaced.

    So the only safe way to access those arguments is to manually use the undocumented funcy.decorator.arggetter directly, or to manually implement the same logic of looking up the argument from call._args and call._kwargs based on the signature gotten from call._func.

    Actually, I think this problem might possibly effect all of the dunder methods too.

    Anyway, I think this could be solved by either:

    1. Just officially saying that as a special case, accessing those arguments to the wrapped function using the same built-in logic for getting arguments is completely unsupported.

    2. Officially documenting funcy.decorator.arggetter as the way to get those arguments.

    3. Using self.__dict__.setdefault instead of self.__dict__[...] in __getattr__, and documenting manually calling call.__getattr__ as the way.

    4. Exposing some sort of new method on the Call class like call._getarg, which could use either the 2nd or 3rd option internally.

    Thoughts?

    opened by mentalisttraceur 19
  • Pipe, thread

    Pipe, thread

    Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

    opened by berrytj 16
  • Feature request: Support method chaining

    Feature request: Support method chaining

    I would find method chaining very useful. This style is made available in many similar libraries, e.g. underscore, lodash, pydash and pandas.

    This is the only feature I've really been missing in funcy.

    opened by taleinat 14
  • Play nicely with static code analyzers

    Play nicely with static code analyzers

    In my case, PyCharm has trouble understanding things coming from funcy. Specifically, it fails to understand importing directly from the funcy package rather than the specific sub-modules, i.e. from funcy import merge causes it to mark the import as problematic. I imagine it has other issues too, I haven't investigated further (yet).

    I can think of several ways to improve the situation:

    1. Re-work the fancy importing mechanisms so that static analyzers do a better job of understanding them.
    2. Include stubs which will directly inform static analyzers about the structure of the funcy library and its contents
    3. Contribute stubs for funcy to the typeshed project, which is used by various static analyzers and IDEs.

    I'd be happy to help with this, if the maintainer(s) are open to it.

    opened by taleinat 14
  • All functions in the library should be curried.

    All functions in the library should be curried.

    I think making all functions from funcy automatically curried would simplify a vast number of use cases. For example, for an extract_int example (where extract_int("Hello, 123! We are 456!") => [123, 456]), without autocurry, the current best way to do this is:

    from funcy import map, partial, rcompose, re_all
    extract_int = rcompose(partial(re_all, r'\d+'), partial(map, int))
    

    Of course, we could use autocurry on them ourselves - but this doesn't seem to work well with some functions (I tested only with map and re_all, surprisingly re_all works):

    >>> from funcy import map, autocurry
    >>> map = autocurry(map)
    >>> map(int)([1, 2, 3])
    # .... blah error stuff ....
    TypeError: map() requires at least two args
    

    This and other such problems could be averted by having all functions (at least those in the library) automatically curried.

    looking for more interest 
    opened by ghost 14
  • Allow specifying an error handler for memoize

    Allow specifying an error handler for memoize

    We ran into an issue where we'd still like an exception to be memoized and return the exception when using the memoize decorator. A way to specify an error handler function would be appreciated. Or provide a list of exception types which should be cached.

    opened by richard-engineering 11
  • Adding a `flip` function to funcy

    Adding a `flip` function to funcy

    Hi,

    every time I try to code in a more functional manner in Python I find myself writing this little function sooner or later:

    
    flip = lambda f: lambda x,y: f(y,x)
    

    it's especially useful with currying, which is already provided by the library, so I think it would be a good addition. A very simple example:

    
    has_suffix = curry(flip(str.endswith))
    filter(has_suffix("a"), ["ab", "ba", "cd"])
    

    One thing to think about is the semantics of flip when used on a function with more than two arguments and on functions with **kwargs, especially the former case is interesting, as it could mean:

    
    flip = lambda f: lambda a,b,*args: f(b,a,*args)
    

    or

    
    flip = lambda f: lambda *args: f(*(args[1:] + [args[0]]))
    

    The latter meaning would be consistent with for example rotatef from CL, but on the other hand it could be less practical.

    opened by piotrklibert 11
  • Add kwargs support for memoize

    Add kwargs support for memoize

    Happy to update docs

    TODO: check performance impact of using inspect.getcallargs (it on its own does fallbacks depending how signature looks like)

    @Suor should I add feature proposal Issue on GH ?

    opened by lukmdo 11
  • `join_with` - transform single elements too

    `join_with` - transform single elements too

    Hi! funcy is a great tool. I especially like join_with because it easily and fast merges many dicts into one in, for example, a defaultdict(list) style. However, I noticed that if, for example, a list of dicts contains only one element - join_with doesn't change its structure. It could be very useful to change them too:

    list_of_dicts = [{1: 2, 3: 4, "foo": "bar"}]
    
    # current implementation
    join_with(list, list_of_dicts) -> {1: 2, 3: 4, "foo": "bar"}
    
    # proposed implementation
    join_with(list, list_of_dicts) -> {1: [2], 3: [4], "foo": ["bar"]}
    
    opened by meh-wzdech 10
  • Join the forces

    Join the forces

    Python has atleast 3 great functional programming libraries. These libraries have lots of overlapping functionality:

    • https://github.com/pytoolz/toolz
    • https://github.com/Suor/funcy
    • https://github.com/kachayev/fn.py

    Maybe also this:

    • https://github.com/Suor/whatever

    I would see as the benefit of the whole community if the authors of all these libraries could work more together on a common codebase (I'm willing to participate also in making this happen).

    @Suor, @pytoolz, @mrocklin, @kachayev Thank you for the hard work all of you have put into these wonderful packages! What do you guys think about this? Could we make more collaboration between the projects?

    opened by kvesteri 10
  • Make get_in, set_in work with lists/indexables

    Make get_in, set_in work with lists/indexables

    Currently:

    def get_in(coll, path, default=None):
        for key in path:
            if key in coll:
                coll = coll[key]
            else:
                return default
        return coll
    

    With current implementation, get_in([1, 2, 3, 4], [0]) returns None, while get_in([1,2,3,4], [1]) returns 2, due to the in behavior, which is slightly funky.

    I've modified get_in and set_in to work with more general indexables - set_in only with lists but get_in with general indexables.

    If this change sounds good, happy to add to docs as well.

    Thanks!

    opened by mcamac 9
  • feat($retry): enable capped exponential backoff and jitter via keyword arguments

    feat($retry): enable capped exponential backoff and jitter via keyword arguments

    Summary

    • Enable exponential backoff via keyword arguments.

    Checklist

    • [x] Implement unit test cases.

    Reason

    • Currently, the feature supports highly customized retry intervals via a pass-in callable argument.
    • However, it's hard to comprehend that the argument timeout can also accept a callable object without tracing the source code.
    • Capped exponential backoffs and jitter are already well-known approaches, maybe you can consider making them specific features via keyword arguments.
    opened by RainrainWu 0
  • funcy.autocurry doesn't work with functions with key-words only

    funcy.autocurry doesn't work with functions with key-words only

    Hi! I'm happy to use the library but looks like I found a bug:

    >>>@funcy.autocurry
    ...def a(b, c): return b + c
    
    >>>a(c=1)(2)
    3
    
    >>>@funcy.autocurry
    ...def aa(b, *, c): return b + c
    
    >>>aa(c=1)
    Traceback (most recent call last):
       ...
    TypeError: aa() missing 1 required keyword-only argument: 'c'
    

    funcy version: 1.17

    opened by ADR-007 2
  • rcurry and similar functions do not work with methods of built-in types

    rcurry and similar functions do not work with methods of built-in types

    >>> has_suffix = rcurry(str.endswith)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\funcs.py", line 57, in rcurry
        n = get_spec(func).max_n
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\_inspect.py", line 108, in get_spec
        mod = func.__module__
    AttributeError: 'method_descriptor' object has no attribute '__module__'. Did you mean: '__reduce__'?```
    looking for more interest 
    opened by DerekForgione 6
  • Add type stubs

    Add type stubs

    Stub files can be added to the project to provide static type information that can be used by tools like mypy.

    I believe that the simplest way to add this functionality to funcy is to provide stub files along with their respective modules. That is, for example, a stub file colls.pyi corresponding to the contents of colls.py; a funcs.pyi corresponding to funcs.py and so on. Corresponding snippets from funcs.py and funcs.pyi would look like:

    funcs.py

    def identity(x):
        return x
    
    def constantly(x):
        return lambda *a, **kw: x
    
    def caller(*a, **kw):
        return lambda f: f(*a, **kw)
    

    funcs.pyi

    from typing import Any, Callable, TypeVar
    
    T = TypeVar('T')
    
    def identity(x: T) -> T: ...
    def constantly(x: T) -> Callable[..., T]: ...
    def caller(*a: Any, **kw: Any) -> Callable[[Callable[..., T]], T]: ...
    

    Take a look at more-itertools to see how this looks like when fully implemented.

    If you are interested in this functionality, I could perhaps submit a draft PR by the end of the week.

    opened by ruancomelli 18
  • get_in() raises TypeError when index is a string but structure is a list

    get_in() raises TypeError when index is a string but structure is a list

    from funcy import get_in
    
    get_in({}, "key")  # example 1
    get_in([], "key")  # example 2
    

    In the code sample above, example 1 returns the default value because the path doesn't exist. However, example 2 fails with:

    .../site-packages/funcy/colls.py in get_in(coll, path, default)
        271     for key in path:
        272         try:
    --> 273             coll = coll[key]
        274         except (KeyError, IndexError):
        275             return default
    
    TypeError: list indices must be integers or slices, not str
    

    This is understandable but to handle this case, a special error handling needs to be added around the function and it kind of defeats the purpose of a simple one-liner. It is also not documented.

    I would be happy to do a PR if you agree get_in() should handle this exception automatically by also returning the default value.

    looking for more interest 
    opened by radeklat 10
Owner
Alexander Schepanovski
Alexander Schepanovski
Cython implementation of Toolz: High performance functional utilities

CyToolz Cython implementation of the toolz package, which provides high performance utility functions for iterables, functions, and dictionaries. tool

null 880 Sep 27, 2022
Functional programming in Python: implementation of missing features to enjoy FP

Fn.py: enjoy FP in Python Despite the fact that Python is not pure-functional programming language, it's multi-paradigm PL and it gives you enough fre

Oleksii Kachaiev 3.2k Sep 30, 2022
A functional standard library for Python.

Toolz A set of utility functions for iterators, functions, and dictionaries. See the PyToolz documentation at https://toolz.readthedocs.io LICENSE New

null 4k Sep 25, 2022
Make your functions return something meaningful, typed, and safe!

Make your functions return something meaningful, typed, and safe! Features Brings functional programming to Python land Provides a bunch of primitives

dry-python 2.4k Oct 1, 2022
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Sep 26, 2022
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Sep 30, 2022
Module for converting 2D Python lists to fancy ASCII tables. Table2Ascii lets you display pretty tables in the terminal and on Discord.

table2ascii Module for converting 2D Python lists to a fancy ASCII/Unicode tables table2ascii ?? Installation ??‍?? Usage Convert lists to ASCII table

Jonah Lawrence 30 Sep 28, 2022
Fancy console logger and wise assistant within your python projects

Fancy console logger and wise assistant within your python projects. Made to save tons of hours for common routines.

BoB 5 Apr 1, 2022
bpython - A fancy curses interface to the Python interactive interpreter

bpython: A fancy curses interface to the Python interactive interpreter bpython is a lightweight Python interpreter that adds several features common

bpython 2.2k Sep 28, 2022
Fancy data functions that will make your life as a data scientist easier.

WhiteBox Utilities Toolkit: Tools to make your life easier Fancy data functions that will make your life as a data scientist easier. Installing To ins

WhiteBox 2 Nov 3, 2021
pubmex.py - a script to get a fancy paper title based on given DOI or PMID

pubmex.py is a script to get a fancy paper title based on given DOI or PMID (can be also combined with macOS Finder)

Marcin Magnus 11 May 10, 2022
Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

English | 简体中文 Introduction PaddleOCR aims to create multilingual, awesome, leading, and practical OCR tools that help users train better models and a

null 25k Sep 26, 2022
Fully functional ecommerce website with user and guest checkout capabilities and Paypal payment integration.

ecommerce_website Fully functional ecommerce website with user and guest checkout capabilities and Paypal payment integration. pip install django pyth

null 2 Jan 5, 2022
💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline!

LocalStack - A fully functional local AWS cloud stack LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. Cur

LocalStack 43.7k Oct 2, 2022
A quick and dirty QT Statusbar implementation for grabbing GIFs from Tenor, since there is no offical or unofficial one I found. This was intended for use under Linux, however it was also functional enough on MacOS.

Statusbar-TenorGIF App for Linux A quick and dirty QT Statusbar implementation for grabbing GIFs from Tenor, since there is no offical one and I didnt

Luigi DaVinci 1 Nov 1, 2021
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

chitra What is chitra? chitra (चित्र) is a multi-functional library for full-stack Deep Learning. It simplifies Model Building, API development, and M

Aniket Maurya 204 Sep 30, 2022
Repository, with small useful and functional applications

Repositorio,com pequenos aplicativos uteis e funcionais A ideia e ir deselvolvendo pequenos aplicativos funcionais e adicionar a este repositorio List

GabrielDuke 6 Dec 6, 2021
A functional and efficient python implementation of the 3D version of Maxwell's equations

py-maxwell-fdfd Solving Maxwell's equations via A python implementation of the 3D curl-curl E-field equations. This code contains additional work to e

Nathan Zhao 11 Aug 16, 2022
Codes of CVPR2022 paper: Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction

Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction Figure 1. Teaser. Introduction This paper studies the problem

Yining Hong 29 Aug 22, 2022
Simple, elegant, Pythonic functional programming.

Coconut Coconut (coconut-lang.org) is a variant of Python that adds on top of Python syntax new features for simple, elegant, Pythonic functional prog

Evan Hubinger 3.6k Oct 1, 2022