A fancy and practical functional tools

Overview

Funcy Build Status Join the chat at https://gitter.im/Suor/funcy

A collection of fancy functional tools focused on practicality.

Inspired by clojure, underscore and my own abstractions. Keep reading to get an overview or read the docs. Or jump directly to cheatsheet.

Works with Python 2.7, 3.4+ and pypy.

Installation

pip install funcy

Overview

Import stuff from funcy to make things happen:

from funcy import whatever, you, need

Merge collections of same type (works for dicts, sets, lists, tuples, iterators and even strings):

merge(coll1, coll2, coll3, ...)
join(colls)
merge_with(sum, dict1, dict2, ...)

Walk through collection, creating its transform (like map but preserves type):

walk(str.upper, {'a', 'b'})            # {'A', 'B'}
walk(reversed, {'a': 1, 'b': 2})       # {1: 'a', 2: 'b'}
walk_keys(double, {'a': 1, 'b': 2})    # {'aa': 1, 'bb': 2}
walk_values(inc, {'a': 1, 'b': 2})     # {'a': 2, 'b': 3}

Select a part of collection:

select(even, {1,2,3,10,20})                  # {2,10,20}
select(r'^a', ('a','b','ab','ba'))           # ('a','ab')
select_keys(callable, {str: '', None: None}) # {str: ''}
compact({2, None, 1, 0})                     # {1,2}

Manipulate sequences:

take(4, iterate(double, 1)) # [1, 2, 4, 8]
first(drop(3, count(10)))   # 13

lremove(even, [1, 2, 3])    # [1, 3]
lconcat([1, 2], [5, 6])     # [1, 2, 5, 6]
lcat(map(range, range(4)))  # [0, 0, 1, 0, 1, 2]
lmapcat(range, range(4))    # same
flatten(nested_structure)   # flat iter
distinct('abacbdd')         # iter('abcd')

lsplit(odd, range(5))       # ([1, 3], [0, 2, 4])
lsplit_at(2, range(5))      # ([0, 1], [2, 3, 4])
group_by(mod3, range(5))    # {0: [0, 3], 1: [1, 4], 2: [2]}

lpartition(2, range(5))     # [[0, 1], [2, 3]]
chunks(2, range(5))         # iter: [0, 1], [2, 3], [4]
pairwise(range(5))          # iter: [0, 1], [1, 2], ...

And functions:

partial(add, 1)             # inc
curry(add)(1)(2)            # 3
compose(inc, double)(10)    # 21
complement(even)            # odd
all_fn(isa(int), even)      # is_even_int

one_third = rpartial(operator.div, 3.0)
has_suffix = rcurry(str.endswith)

Create decorators easily:

@decorator
def log(call):
    print call._func.__name__, call._args
    return call()

Abstract control flow:

walk_values(silent(int), {'a': '1', 'b': 'no'})
# => {'a': 1, 'b': None}

@once
def initialize():
    "..."

with suppress(OSError):
    os.remove('some.file')

@ignore(ErrorRateExceeded)
@limit_error_rate(fails=5, timeout=60)
@retry(tries=2, errors=(HttpError, ServiceDown))
def some_unreliable_action(...):
    "..."

class MyUser(AbstractBaseUser):
    @cached_property
    def public_phones(self):
        return self.phones.filter(public=True)

Ease debugging:

squares = {tap(x, 'x'): tap(x * x, 'x^2') for x in [3, 4]}
# x: 3
# x^2: 9
# ...

@print_exits
def some_func(...):
    "..."

@log_calls(log.info, errors=False)
@log_errors(log.exception)
def some_suspicious_function(...):
    "..."

with print_durations('Creating models'):
    Model.objects.create(...)
    # ...
# 10.2 ms in Creating models

And much more.

Dive in

Funcy is an embodiment of ideas I explain in several essays:

Running tests

To run the tests using your default python:

pip install -r test_requirements.txt
py.test

To fully run tox you need all the supported pythons to be installed. These are 2.6+, 3.3+, PyPy and PyPy3. You can run it for particular environment even in absense of all of the above:

tox -e py27
tox -e py36
tox -e lint
Comments
  • Argument collision with @decorator

    Argument collision with @decorator

    Logically obvious limitation without an obviously documented solution.

    Suppose a function decorated by a decorator created with @funcy.decorator actually has arguments named _args, _kwargs, or _func (it shouldn't, but supposing it did).

    Reasonably and correctly, call._args, call._kwargs, and call._func still resolve as expected and documented - good so far.

    An intuitive solution if you did want to inspect those arguments would be to call @decorator's __getattr__(self, ...) manually.

    But that unconditionally sets self.__dict__[...], so for example doing call.__getattr__('_args') can break call(), because call._args gets replaced.

    So the only safe way to access those arguments is to manually use the undocumented funcy.decorator.arggetter directly, or to manually implement the same logic of looking up the argument from call._args and call._kwargs based on the signature gotten from call._func.

    Actually, I think this problem might possibly effect all of the dunder methods too.

    Anyway, I think this could be solved by either:

    1. Just officially saying that as a special case, accessing those arguments to the wrapped function using the same built-in logic for getting arguments is completely unsupported.

    2. Officially documenting funcy.decorator.arggetter as the way to get those arguments.

    3. Using self.__dict__.setdefault instead of self.__dict__[...] in __getattr__, and documenting manually calling call.__getattr__ as the way.

    4. Exposing some sort of new method on the Call class like call._getarg, which could use either the 2nd or 3rd option internally.

    Thoughts?

    opened by mentalisttraceur 19
  • Pipe, thread

    Pipe, thread

    Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

    opened by berrytj 16
  • Feature request: Support method chaining

    Feature request: Support method chaining

    I would find method chaining very useful. This style is made available in many similar libraries, e.g. underscore, lodash, pydash and pandas.

    This is the only feature I've really been missing in funcy.

    opened by taleinat 14
  • Play nicely with static code analyzers

    Play nicely with static code analyzers

    In my case, PyCharm has trouble understanding things coming from funcy. Specifically, it fails to understand importing directly from the funcy package rather than the specific sub-modules, i.e. from funcy import merge causes it to mark the import as problematic. I imagine it has other issues too, I haven't investigated further (yet).

    I can think of several ways to improve the situation:

    1. Re-work the fancy importing mechanisms so that static analyzers do a better job of understanding them.
    2. Include stubs which will directly inform static analyzers about the structure of the funcy library and its contents
    3. Contribute stubs for funcy to the typeshed project, which is used by various static analyzers and IDEs.

    I'd be happy to help with this, if the maintainer(s) are open to it.

    opened by taleinat 14
  • All functions in the library should be curried.

    All functions in the library should be curried.

    I think making all functions from funcy automatically curried would simplify a vast number of use cases. For example, for an extract_int example (where extract_int("Hello, 123! We are 456!") => [123, 456]), without autocurry, the current best way to do this is:

    from funcy import map, partial, rcompose, re_all
    extract_int = rcompose(partial(re_all, r'\d+'), partial(map, int))
    

    Of course, we could use autocurry on them ourselves - but this doesn't seem to work well with some functions (I tested only with map and re_all, surprisingly re_all works):

    >>> from funcy import map, autocurry
    >>> map = autocurry(map)
    >>> map(int)([1, 2, 3])
    # .... blah error stuff ....
    TypeError: map() requires at least two args
    

    This and other such problems could be averted by having all functions (at least those in the library) automatically curried.

    looking for more interest 
    opened by ghost 14
  • Allow specifying an error handler for memoize

    Allow specifying an error handler for memoize

    We ran into an issue where we'd still like an exception to be memoized and return the exception when using the memoize decorator. A way to specify an error handler function would be appreciated. Or provide a list of exception types which should be cached.

    opened by richard-engineering 11
  • Adding a `flip` function to funcy

    Adding a `flip` function to funcy

    Hi,

    every time I try to code in a more functional manner in Python I find myself writing this little function sooner or later:

    
    flip = lambda f: lambda x,y: f(y,x)
    

    it's especially useful with currying, which is already provided by the library, so I think it would be a good addition. A very simple example:

    
    has_suffix = curry(flip(str.endswith))
    filter(has_suffix("a"), ["ab", "ba", "cd"])
    

    One thing to think about is the semantics of flip when used on a function with more than two arguments and on functions with **kwargs, especially the former case is interesting, as it could mean:

    
    flip = lambda f: lambda a,b,*args: f(b,a,*args)
    

    or

    
    flip = lambda f: lambda *args: f(*(args[1:] + [args[0]]))
    

    The latter meaning would be consistent with for example rotatef from CL, but on the other hand it could be less practical.

    opened by piotrklibert 11
  • Add kwargs support for memoize

    Add kwargs support for memoize

    Happy to update docs

    TODO: check performance impact of using inspect.getcallargs (it on its own does fallbacks depending how signature looks like)

    @Suor should I add feature proposal Issue on GH ?

    opened by lukmdo 11
  • `join_with` - transform single elements too

    `join_with` - transform single elements too

    Hi! funcy is a great tool. I especially like join_with because it easily and fast merges many dicts into one in, for example, a defaultdict(list) style. However, I noticed that if, for example, a list of dicts contains only one element - join_with doesn't change its structure. It could be very useful to change them too:

    list_of_dicts = [{1: 2, 3: 4, "foo": "bar"}]
    
    # current implementation
    join_with(list, list_of_dicts) -> {1: 2, 3: 4, "foo": "bar"}
    
    # proposed implementation
    join_with(list, list_of_dicts) -> {1: [2], 3: [4], "foo": ["bar"]}
    
    opened by meh-wzdech 10
  • Join the forces

    Join the forces

    Python has atleast 3 great functional programming libraries. These libraries have lots of overlapping functionality:

    • https://github.com/pytoolz/toolz
    • https://github.com/Suor/funcy
    • https://github.com/kachayev/fn.py

    Maybe also this:

    • https://github.com/Suor/whatever

    I would see as the benefit of the whole community if the authors of all these libraries could work more together on a common codebase (I'm willing to participate also in making this happen).

    @Suor, @pytoolz, @mrocklin, @kachayev Thank you for the hard work all of you have put into these wonderful packages! What do you guys think about this? Could we make more collaboration between the projects?

    opened by kvesteri 10
  • Make get_in, set_in work with lists/indexables

    Make get_in, set_in work with lists/indexables

    Currently:

    def get_in(coll, path, default=None):
        for key in path:
            if key in coll:
                coll = coll[key]
            else:
                return default
        return coll
    

    With current implementation, get_in([1, 2, 3, 4], [0]) returns None, while get_in([1,2,3,4], [1]) returns 2, due to the in behavior, which is slightly funky.

    I've modified get_in and set_in to work with more general indexables - set_in only with lists but get_in with general indexables.

    If this change sounds good, happy to add to docs as well.

    Thanks!

    opened by mcamac 9
  • Unprecise documentation on extended function semantics

    Unprecise documentation on extended function semantics

    I stumbled mentions on extended function semantics and went to this section of the doc: https://funcy.readthedocs.io/en/stable/extended_fns.html#extended-fns

    IMHO, this section could use some examples. For example it states that providing a mapping instead of a callable would result in lambda x: f[x] being used. It took me some time to figure out how to use this just by reading this dry description.

    Would you be interested that I complete this page a bit with usage examples?

    opened by antoine-gallix 1
  • feat($retry): enable capped exponential backoff and jitter via keyword arguments

    feat($retry): enable capped exponential backoff and jitter via keyword arguments

    Summary

    • Enable exponential backoff via keyword arguments.

    Checklist

    • [x] Implement unit test cases.

    Reason

    • Currently, the feature supports highly customized retry intervals via a pass-in callable argument.
    • However, it's hard to comprehend that the argument timeout can also accept a callable object without tracing the source code.
    • Capped exponential backoffs and jitter are already well-known approaches, maybe you can consider making them specific features via keyword arguments.
    opened by RainrainWu 0
  • funcy.autocurry doesn't work with functions with key-words only

    funcy.autocurry doesn't work with functions with key-words only

    Hi! I'm happy to use the library but looks like I found a bug:

    >>>@funcy.autocurry
    ...def a(b, c): return b + c
    
    >>>a(c=1)(2)
    3
    
    >>>@funcy.autocurry
    ...def aa(b, *, c): return b + c
    
    >>>aa(c=1)
    Traceback (most recent call last):
       ...
    TypeError: aa() missing 1 required keyword-only argument: 'c'
    

    funcy version: 1.17

    opened by ADR-007 2
  • rcurry and similar functions do not work with methods of built-in types

    rcurry and similar functions do not work with methods of built-in types

    >>> has_suffix = rcurry(str.endswith)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\funcs.py", line 57, in rcurry
        n = get_spec(func).max_n
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\_inspect.py", line 108, in get_spec
        mod = func.__module__
    AttributeError: 'method_descriptor' object has no attribute '__module__'. Did you mean: '__reduce__'?```
    looking for more interest 
    opened by NullSurplus 6
  • Add type stubs

    Add type stubs

    Stub files can be added to the project to provide static type information that can be used by tools like mypy.

    I believe that the simplest way to add this functionality to funcy is to provide stub files along with their respective modules. That is, for example, a stub file colls.pyi corresponding to the contents of colls.py; a funcs.pyi corresponding to funcs.py and so on. Corresponding snippets from funcs.py and funcs.pyi would look like:

    funcs.py

    def identity(x):
        return x
    
    def constantly(x):
        return lambda *a, **kw: x
    
    def caller(*a, **kw):
        return lambda f: f(*a, **kw)
    

    funcs.pyi

    from typing import Any, Callable, TypeVar
    
    T = TypeVar('T')
    
    def identity(x: T) -> T: ...
    def constantly(x: T) -> Callable[..., T]: ...
    def caller(*a: Any, **kw: Any) -> Callable[[Callable[..., T]], T]: ...
    

    Take a look at more-itertools to see how this looks like when fully implemented.

    If you are interested in this functionality, I could perhaps submit a draft PR by the end of the week.

    opened by ruancomelli 18
Owner
Alexander Schepanovski
Alexander Schepanovski
Cython implementation of Toolz: High performance functional utilities

CyToolz Cython implementation of the toolz package, which provides high performance utility functions for iterables, functions, and dictionaries. tool

null 894 Jan 2, 2023
Functional programming in Python: implementation of missing features to enjoy FP

Fn.py: enjoy FP in Python Despite the fact that Python is not pure-functional programming language, it's multi-paradigm PL and it gives you enough fre

Oleksii Kachaiev 3.3k Jan 4, 2023
A functional standard library for Python.

Toolz A set of utility functions for iterators, functions, and dictionaries. See the PyToolz documentation at https://toolz.readthedocs.io LICENSE New

null 4.1k Jan 3, 2023
Make your functions return something meaningful, typed, and safe!

Make your functions return something meaningful, typed, and safe! Features Brings functional programming to Python land Provides a bunch of primitives

dry-python 2.5k Jan 5, 2023
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Dec 29, 2022
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Jan 7, 2023
Module for converting 2D Python lists to fancy ASCII tables. Table2Ascii lets you display pretty tables in the terminal and on Discord.

table2ascii Module for converting 2D Python lists to a fancy ASCII/Unicode tables table2ascii ?? Installation ??‍?? Usage Convert lists to ASCII table

Jonah Lawrence 40 Jan 3, 2023
Fancy console logger and wise assistant within your python projects

Fancy console logger and wise assistant within your python projects. Made to save tons of hours for common routines.

BoB 5 Apr 1, 2022
bpython - A fancy curses interface to the Python interactive interpreter

bpython: A fancy curses interface to the Python interactive interpreter bpython is a lightweight Python interpreter that adds several features common

bpython 2.2k Dec 27, 2022
Fancy data functions that will make your life as a data scientist easier.

WhiteBox Utilities Toolkit: Tools to make your life easier Fancy data functions that will make your life as a data scientist easier. Installing To ins

WhiteBox 3 Oct 3, 2022
pubmex.py - a script to get a fancy paper title based on given DOI or PMID

pubmex.py is a script to get a fancy paper title based on given DOI or PMID (can be also combined with macOS Finder)

Marcin Magnus 13 Nov 20, 2022
Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

English | 简体中文 Introduction PaddleOCR aims to create multilingual, awesome, leading, and practical OCR tools that help users train better models and a

null 27.5k Jan 8, 2023
All the essential resources and template code needed to understand and practice data structures and algorithms in python with few small projects to demonstrate their practical application.

Data Structures and Algorithms Python INDEX 1. Resources - Books Data Structures - Reema Thareja competitiveCoding Big-O Cheat Sheet DAA Syllabus Inte

Shushrut Kumar 129 Dec 15, 2022
Drug design and development team HackBio internship is a virtual bioinformatics program that introduces students and professional to advanced practical bioinformatics and its applications globally.

-Nyokong. Drug design and development team HackBio internship is a virtual bioinformatics program that introduces students and professional to advance

null 4 Aug 4, 2022
Fully functional ecommerce website with user and guest checkout capabilities and Paypal payment integration.

ecommerce_website Fully functional ecommerce website with user and guest checkout capabilities and Paypal payment integration. pip install django pyth

null 2 Jan 5, 2022
A scientific and useful toolbox, which contains practical and effective long-tail related tricks with extensive experimental results

Bag of tricks for long-tailed visual recognition with deep convolutional neural networks This repository is the official PyTorch implementation of AAA

Yong-Shun Zhang 181 Dec 28, 2022
This project is based on RIFE and aims to make RIFE more practical for users by adding various features and design new models

This project is based on RIFE and aims to make RIFE more practical for users by adding various features and design new models. Because improving the PSNR index is not compatible with subjective effects, we hope this part of work and our academic research are independent of each other.

hzwer 190 Jan 8, 2023
Prithivida 690 Jan 4, 2023
A collection of neat and practical data science and machine learning projects

Data Science A collection of neat and practical data science and machine learning projects Explore the docs » Report Bug · Request Feature Table of Co

Will Fong 2 Dec 10, 2021
Adaout is a practical and flexible regularization method with high generalization and interpretability

Adaout Adaout is a practical and flexible regularization method with high generalization and interpretability. Requirements python 3.6 (Anaconda versi

lambett 1 Feb 9, 2022