A fancy and practical functional tools

Overview

Funcy Build Status Join the chat at https://gitter.im/Suor/funcy

A collection of fancy functional tools focused on practicality.

Inspired by clojure, underscore and my own abstractions. Keep reading to get an overview or read the docs. Or jump directly to cheatsheet.

Works with Python 2.7, 3.4+ and pypy.

Installation

pip install funcy

Overview

Import stuff from funcy to make things happen:

from funcy import whatever, you, need

Merge collections of same type (works for dicts, sets, lists, tuples, iterators and even strings):

merge(coll1, coll2, coll3, ...)
join(colls)
merge_with(sum, dict1, dict2, ...)

Walk through collection, creating its transform (like map but preserves type):

walk(str.upper, {'a', 'b'})            # {'A', 'B'}
walk(reversed, {'a': 1, 'b': 2})       # {1: 'a', 2: 'b'}
walk_keys(double, {'a': 1, 'b': 2})    # {'aa': 1, 'bb': 2}
walk_values(inc, {'a': 1, 'b': 2})     # {'a': 2, 'b': 3}

Select a part of collection:

select(even, {1,2,3,10,20})                  # {2,10,20}
select(r'^a', ('a','b','ab','ba'))           # ('a','ab')
select_keys(callable, {str: '', None: None}) # {str: ''}
compact({2, None, 1, 0})                     # {1,2}

Manipulate sequences:

take(4, iterate(double, 1)) # [1, 2, 4, 8]
first(drop(3, count(10)))   # 13

lremove(even, [1, 2, 3])    # [1, 3]
lconcat([1, 2], [5, 6])     # [1, 2, 5, 6]
lcat(map(range, range(4)))  # [0, 0, 1, 0, 1, 2]
lmapcat(range, range(4))    # same
flatten(nested_structure)   # flat iter
distinct('abacbdd')         # iter('abcd')

lsplit(odd, range(5))       # ([1, 3], [0, 2, 4])
lsplit_at(2, range(5))      # ([0, 1], [2, 3, 4])
group_by(mod3, range(5))    # {0: [0, 3], 1: [1, 4], 2: [2]}

lpartition(2, range(5))     # [[0, 1], [2, 3]]
chunks(2, range(5))         # iter: [0, 1], [2, 3], [4]
pairwise(range(5))          # iter: [0, 1], [1, 2], ...

And functions:

partial(add, 1)             # inc
curry(add)(1)(2)            # 3
compose(inc, double)(10)    # 21
complement(even)            # odd
all_fn(isa(int), even)      # is_even_int

one_third = rpartial(operator.div, 3.0)
has_suffix = rcurry(str.endswith)

Create decorators easily:

@decorator
def log(call):
    print call._func.__name__, call._args
    return call()

Abstract control flow:

walk_values(silent(int), {'a': '1', 'b': 'no'})
# => {'a': 1, 'b': None}

@once
def initialize():
    "..."

with suppress(OSError):
    os.remove('some.file')

@ignore(ErrorRateExceeded)
@limit_error_rate(fails=5, timeout=60)
@retry(tries=2, errors=(HttpError, ServiceDown))
def some_unreliable_action(...):
    "..."

class MyUser(AbstractBaseUser):
    @cached_property
    def public_phones(self):
        return self.phones.filter(public=True)

Ease debugging:

squares = {tap(x, 'x'): tap(x * x, 'x^2') for x in [3, 4]}
# x: 3
# x^2: 9
# ...

@print_exits
def some_func(...):
    "..."

@log_calls(log.info, errors=False)
@log_errors(log.exception)
def some_suspicious_function(...):
    "..."

with print_durations('Creating models'):
    Model.objects.create(...)
    # ...
# 10.2 ms in Creating models

And much more.

Dive in

Funcy is an embodiment of ideas I explain in several essays:

Running tests

To run the tests using your default python:

pip install -r test_requirements.txt
py.test

To fully run tox you need all the supported pythons to be installed. These are 2.6+, 3.3+, PyPy and PyPy3. You can run it for particular environment even in absense of all of the above:

tox -e py27
tox -e py36
tox -e lint
Comments
  • Argument collision with @decorator

    Argument collision with @decorator

    Logically obvious limitation without an obviously documented solution.

    Suppose a function decorated by a decorator created with @funcy.decorator actually has arguments named _args, _kwargs, or _func (it shouldn't, but supposing it did).

    Reasonably and correctly, call._args, call._kwargs, and call._func still resolve as expected and documented - good so far.

    An intuitive solution if you did want to inspect those arguments would be to call @decorator's __getattr__(self, ...) manually.

    But that unconditionally sets self.__dict__[...], so for example doing call.__getattr__('_args') can break call(), because call._args gets replaced.

    So the only safe way to access those arguments is to manually use the undocumented funcy.decorator.arggetter directly, or to manually implement the same logic of looking up the argument from call._args and call._kwargs based on the signature gotten from call._func.

    Actually, I think this problem might possibly effect all of the dunder methods too.

    Anyway, I think this could be solved by either:

    1. Just officially saying that as a special case, accessing those arguments to the wrapped function using the same built-in logic for getting arguments is completely unsupported.

    2. Officially documenting funcy.decorator.arggetter as the way to get those arguments.

    3. Using self.__dict__.setdefault instead of self.__dict__[...] in __getattr__, and documenting manually calling call.__getattr__ as the way.

    4. Exposing some sort of new method on the Call class like call._getarg, which could use either the 2nd or 3rd option internally.

    Thoughts?

    opened by mentalisttraceur 19
  • Pipe, thread

    Pipe, thread

    Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

    opened by berrytj 16
  • Feature request: Support method chaining

    Feature request: Support method chaining

    I would find method chaining very useful. This style is made available in many similar libraries, e.g. underscore, lodash, pydash and pandas.

    This is the only feature I've really been missing in funcy.

    opened by taleinat 14
  • Play nicely with static code analyzers

    Play nicely with static code analyzers

    In my case, PyCharm has trouble understanding things coming from funcy. Specifically, it fails to understand importing directly from the funcy package rather than the specific sub-modules, i.e. from funcy import merge causes it to mark the import as problematic. I imagine it has other issues too, I haven't investigated further (yet).

    I can think of several ways to improve the situation:

    1. Re-work the fancy importing mechanisms so that static analyzers do a better job of understanding them.
    2. Include stubs which will directly inform static analyzers about the structure of the funcy library and its contents
    3. Contribute stubs for funcy to the typeshed project, which is used by various static analyzers and IDEs.

    I'd be happy to help with this, if the maintainer(s) are open to it.

    opened by taleinat 14
  • All functions in the library should be curried.

    All functions in the library should be curried.

    I think making all functions from funcy automatically curried would simplify a vast number of use cases. For example, for an extract_int example (where extract_int("Hello, 123! We are 456!") => [123, 456]), without autocurry, the current best way to do this is:

    from funcy import map, partial, rcompose, re_all
    extract_int = rcompose(partial(re_all, r'\d+'), partial(map, int))
    

    Of course, we could use autocurry on them ourselves - but this doesn't seem to work well with some functions (I tested only with map and re_all, surprisingly re_all works):

    >>> from funcy import map, autocurry
    >>> map = autocurry(map)
    >>> map(int)([1, 2, 3])
    # .... blah error stuff ....
    TypeError: map() requires at least two args
    

    This and other such problems could be averted by having all functions (at least those in the library) automatically curried.

    looking for more interest 
    opened by ghost 14
  • Allow specifying an error handler for memoize

    Allow specifying an error handler for memoize

    We ran into an issue where we'd still like an exception to be memoized and return the exception when using the memoize decorator. A way to specify an error handler function would be appreciated. Or provide a list of exception types which should be cached.

    opened by richard-engineering 11
  • Adding a `flip` function to funcy

    Adding a `flip` function to funcy

    Hi,

    every time I try to code in a more functional manner in Python I find myself writing this little function sooner or later:

    
    flip = lambda f: lambda x,y: f(y,x)
    

    it's especially useful with currying, which is already provided by the library, so I think it would be a good addition. A very simple example:

    
    has_suffix = curry(flip(str.endswith))
    filter(has_suffix("a"), ["ab", "ba", "cd"])
    

    One thing to think about is the semantics of flip when used on a function with more than two arguments and on functions with **kwargs, especially the former case is interesting, as it could mean:

    
    flip = lambda f: lambda a,b,*args: f(b,a,*args)
    

    or

    
    flip = lambda f: lambda *args: f(*(args[1:] + [args[0]]))
    

    The latter meaning would be consistent with for example rotatef from CL, but on the other hand it could be less practical.

    opened by piotrklibert 11
  • Add kwargs support for memoize

    Add kwargs support for memoize

    Happy to update docs

    TODO: check performance impact of using inspect.getcallargs (it on its own does fallbacks depending how signature looks like)

    @Suor should I add feature proposal Issue on GH ?

    opened by lukmdo 11
  • `join_with` - transform single elements too

    `join_with` - transform single elements too

    Hi! funcy is a great tool. I especially like join_with because it easily and fast merges many dicts into one in, for example, a defaultdict(list) style. However, I noticed that if, for example, a list of dicts contains only one element - join_with doesn't change its structure. It could be very useful to change them too:

    list_of_dicts = [{1: 2, 3: 4, "foo": "bar"}]
    
    # current implementation
    join_with(list, list_of_dicts) -> {1: 2, 3: 4, "foo": "bar"}
    
    # proposed implementation
    join_with(list, list_of_dicts) -> {1: [2], 3: [4], "foo": ["bar"]}
    
    opened by meh-wzdech 10
  • Join the forces

    Join the forces

    Python has atleast 3 great functional programming libraries. These libraries have lots of overlapping functionality:

    • https://github.com/pytoolz/toolz
    • https://github.com/Suor/funcy
    • https://github.com/kachayev/fn.py

    Maybe also this:

    • https://github.com/Suor/whatever

    I would see as the benefit of the whole community if the authors of all these libraries could work more together on a common codebase (I'm willing to participate also in making this happen).

    @Suor, @pytoolz, @mrocklin, @kachayev Thank you for the hard work all of you have put into these wonderful packages! What do you guys think about this? Could we make more collaboration between the projects?

    opened by kvesteri 10
  • Make get_in, set_in work with lists/indexables

    Make get_in, set_in work with lists/indexables

    Currently:

    def get_in(coll, path, default=None):
        for key in path:
            if key in coll:
                coll = coll[key]
            else:
                return default
        return coll
    

    With current implementation, get_in([1, 2, 3, 4], [0]) returns None, while get_in([1,2,3,4], [1]) returns 2, due to the in behavior, which is slightly funky.

    I've modified get_in and set_in to work with more general indexables - set_in only with lists but get_in with general indexables.

    If this change sounds good, happy to add to docs as well.

    Thanks!

    opened by mcamac 9
  • Unprecise documentation on extended function semantics

    Unprecise documentation on extended function semantics

    I stumbled mentions on extended function semantics and went to this section of the doc: https://funcy.readthedocs.io/en/stable/extended_fns.html#extended-fns

    IMHO, this section could use some examples. For example it states that providing a mapping instead of a callable would result in lambda x: f[x] being used. It took me some time to figure out how to use this just by reading this dry description.

    Would you be interested that I complete this page a bit with usage examples?

    opened by antoine-gallix 1
  • feat($retry): enable capped exponential backoff and jitter via keyword arguments

    feat($retry): enable capped exponential backoff and jitter via keyword arguments

    Summary

    • Enable exponential backoff via keyword arguments.

    Checklist

    • [x] Implement unit test cases.

    Reason

    • Currently, the feature supports highly customized retry intervals via a pass-in callable argument.
    • However, it's hard to comprehend that the argument timeout can also accept a callable object without tracing the source code.
    • Capped exponential backoffs and jitter are already well-known approaches, maybe you can consider making them specific features via keyword arguments.
    opened by RainrainWu 0
  • funcy.autocurry doesn't work with functions with key-words only

    funcy.autocurry doesn't work with functions with key-words only

    Hi! I'm happy to use the library but looks like I found a bug:

    >>>@funcy.autocurry
    ...def a(b, c): return b + c
    
    >>>a(c=1)(2)
    3
    
    >>>@funcy.autocurry
    ...def aa(b, *, c): return b + c
    
    >>>aa(c=1)
    Traceback (most recent call last):
       ...
    TypeError: aa() missing 1 required keyword-only argument: 'c'
    

    funcy version: 1.17

    opened by ADR-007 2
  • rcurry and similar functions do not work with methods of built-in types

    rcurry and similar functions do not work with methods of built-in types

    >>> has_suffix = rcurry(str.endswith)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\funcs.py", line 57, in rcurry
        n = get_spec(func).max_n
      File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\funcy\_inspect.py", line 108, in get_spec
        mod = func.__module__
    AttributeError: 'method_descriptor' object has no attribute '__module__'. Did you mean: '__reduce__'?```
    looking for more interest 
    opened by NullSurplus 6
  • Add type stubs

    Add type stubs

    Stub files can be added to the project to provide static type information that can be used by tools like mypy.

    I believe that the simplest way to add this functionality to funcy is to provide stub files along with their respective modules. That is, for example, a stub file colls.pyi corresponding to the contents of colls.py; a funcs.pyi corresponding to funcs.py and so on. Corresponding snippets from funcs.py and funcs.pyi would look like:

    funcs.py

    def identity(x):
        return x
    
    def constantly(x):
        return lambda *a, **kw: x
    
    def caller(*a, **kw):
        return lambda f: f(*a, **kw)
    

    funcs.pyi

    from typing import Any, Callable, TypeVar
    
    T = TypeVar('T')
    
    def identity(x: T) -> T: ...
    def constantly(x: T) -> Callable[..., T]: ...
    def caller(*a: Any, **kw: Any) -> Callable[[Callable[..., T]], T]: ...
    

    Take a look at more-itertools to see how this looks like when fully implemented.

    If you are interested in this functionality, I could perhaps submit a draft PR by the end of the week.

    opened by ruancomelli 18
Owner
Alexander Schepanovski
Alexander Schepanovski
A functional standard library for Python.

Toolz A set of utility functions for iterators, functions, and dictionaries. See the PyToolz documentation at https://toolz.readthedocs.io LICENSE New

null 4.1k Jan 4, 2023
a simple functional programming language compiler written in python

Functional Programming Language A compiler for my small functional language. Written in python with SLY lexer/parser generator library. Requirements p

Ashkan Laei 3 Nov 5, 2021
This is a library to do functional programming in Python.

Fpylib This is a library to do functional programming in Python. Index Fpylib Index Features Intelligents Ranges with irange Lazyness to functions Com

Fabián Vega Alcota 4 Jul 17, 2022
Functional collections extension functions for Python

pyfuncol pyfuncol Installation Usage API Documentation Compatibility Contributing License A Python functional collections library. It extends collecti

Andrea Veneziano 32 Nov 16, 2022
Functional interface for concurrent futures, including asynchronous I/O.

Futured provides a consistent interface for concurrent functional programming in Python. It wraps any callable to return a concurrent.futures.Future,

A. Coady 11 Nov 27, 2022
A collection of examples of using cocotb for functional verification of VHDL designs with GHDL.

At the moment, this repo is in an early state and serves as a learning tool for me. So it contains a a lot of quirks and code which can be done much better by cocotb-professionals.

T. Meissner 7 Mar 10, 2022
A simple but fully functional calculator that will take multiple operations.

Functional-Calculator A simple but fully functional calculator that will take multiple operations. Usage Run the following command through terminal: p

Uzziel Ariel 1 Dec 22, 2022
《practical python programming》的中文翻译

欢迎光临 大约 25 年前,当我第一次学习 Python 时,发现 Python 竟然可以被高效地应用到各种混乱的工作项目上,我立即被震惊了。15 年前,我自己也将这种乐趣教授给别人。教学的结果就是本课程——一门实用的学习 Python的课程。

编程人 125 Dec 17, 2022
A tutorial presents several practical examples of how to build DAGs in Apache Airflow

Apache Airflow - Python Brasil 2021 Este tutorial apresenta vários exemplos práticos de como construir DAGs no Apache Airflow. Background Apache Airfl

Jusbrasil 14 Jun 3, 2022
x-tools is a collection of tools developed in Python

x-tools X-tools is a collection of tools developed in Python Commands\

null 5 Jan 24, 2022
A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Ubelt is a small library of robust, tested, documented, and simple functions that extend the Python standard library. It has a flat API that all behav

Jon Crall 638 Dec 13, 2022
null 1 May 12, 2022
🏆 A ranked list of awesome Python open-source libraries and tools. Updated weekly.

Best-of Python ?? A ranked list of awesome Python open-source libraries & tools. Updated weekly. This curated list contains 230 awesome open-source pr

Machine Learning Tooling 2.7k Jan 3, 2023
Some ideas and tools to develop Python 3.8 plugins for GIMP 2.99.4

gimp-python-development Some ideas and tools to develop Python 3.8 plugins for GIMP 2.99.4. GIMP 2.99.4 is the latest unstable pre-release of GIMP 3.

Ismael Benito 53 Sep 25, 2022
Socorro is the Mozilla crash ingestion pipeline. It accepts and processes Breakpad-style crash reports. It provides analysis tools.

Socorro Socorro is a Mozilla-centric ingestion pipeline and analysis tools for crash reports using the Breakpad libraries. Support This is a Mozilla-s

Mozilla Services 552 Dec 19, 2022
addon for blender to import mocap data from tools like easymocap, frankmocap and Vibe

b3d_mocap_import addon for blender to import mocap data from tools like easymocap, frankmocap and Vibe ==================VIBE================== To use

Carlos Barreto 97 Dec 7, 2022
Scientific color maps and standardization tools

Scicomap is a package that provides scientific color maps and tools to standardize your favourite color maps if you don't like the built-in ones. Scicomap currently provides sequential, bi-sequential, diverging, circular, qualitative and miscellaneous color maps. You can easily draw examples, compare the rendering, see how colorblind people will perceive the color maps. I will illustrate the scicomap capabilities below.

Thomas Bury 14 Nov 30, 2022
Unofficial Valorant documentation and tools for third party developers

Valorant Third Party Toolkit This repository contains unofficial Valorant documentation and tools for third party developers. Our goal is to centraliz

Noah Kim 20 Dec 21, 2022
A beacon generator using Cobalt Strike and a variety of tools.

Beaconator is an aggressor script for Cobalt Strike used to generate either staged or stageless shellcode and packing the generated shellcode using your tool of choice.

Capt. Meelo 441 Dec 17, 2022