Cython implementation of Toolz: High performance functional utilities

Overview

CyToolz

Build Status Version Status

Cython implementation of the toolz package, which provides high performance utility functions for iterables, functions, and dictionaries.

toolz is a pure Python package that borrows heavily from contemporary functional languanges. It is designed to interoperate seamlessly with other libraries including itertools, functools, and third party libraries. High performance functional data analysis is possible with builtin types like list and dict, and user-defined data structures; and low memory usage is achieved by using the iterator protocol and returning iterators whenever possible.

cytoolz implements the same API as toolz. The main differences are that cytoolz is faster (typically 2-5x faster with a few spectacular exceptions) and cytoolz offers a C API that is accessible to other projects developed in Cython. Since toolz is able to process very large (potentially infinite) data sets, the performance increase gained by using cytoolz can be significant.

See the PyToolz documentation at https://toolz.readthedocs.io and the full API Documentation for more details.

LICENSE

New BSD. See License File.

Install

cytoolz is on the Python Package Index (PyPI):

pip install cytoolz

Dependencies

cytoolz supports Python 3.5+ with a common codebase. It is developed in Cython, but requires no dependecies other than CPython and a C compiler. Like toolz, it is a light weight dependency.

Contributions Welcome

toolz (and cytoolz) aims to be a repository for utility functions, particularly those that come from the functional programming and list processing traditions. We welcome contributions that fall within this scope and encourage users to scrape their util.py files for functions that are broadly useful.

Please take a look at our issue pages for toolz and cytoolz for contribution ideas.

Community

See our mailing list. We're friendly.

Comments
  • add join

    add join

    This is ugly and likely inefficient. Help.

    toolz.join has some pretty complex yield stuff going on. The work here is pretty much just writing down state explicitly to handle all of that. It's pretty ugly and I assume that it's not any more efficient than the Pure Python solution. Tips appreciated.

    opened by mrocklin 28
  • Problems installing cytoolz on python@3.10.0

    Problems installing cytoolz on [email protected]

    1. python version 3.10.0
    2. pip version 21.3.1

    Running pip install cytoolz is giving me the following error log:

      Using cached cytoolz-0.11.0.tar.gz (477 kB)
      Preparing metadata (setup.py) ... done
    Requirement already satisfied: toolz>=0.8.0 in ./env/lib/python3.10/site-packages (from cytoolz) (0.11.1)
    Building wheels for collected packages: cytoolz
      Building wheel for cytoolz (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: /Users/hristotodorov/Documents/Vyper/env/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"'; __file__='"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-wheel-czxvyqb8
           cwd: /private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/
      Complete output (75 lines):
      ALERT: Cython not installed.  Building without Cython.
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-universal2-3.10
      creating build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/compatibility.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/utils_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/_version.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/__init__.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/_signatures.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      creating build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
      copying cytoolz/curried/operator.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
      copying cytoolz/curried/__init__.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
      copying cytoolz/curried/exceptions.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
      copying cytoolz/itertoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/dicttoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/functoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/recipes.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/utils.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/utils.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/__init__.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/recipes.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/functoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/dicttoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/cpython.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      copying cytoolz/itertoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
      creating build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_none_safe.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_utils.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_curried.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_compatibility.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_embedded_sigs.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_functoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_inspect_args.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_doctests.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_tlz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_signatures.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/dev_skip_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_recipes.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_docstrings.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_dev_skip_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_dicttoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_serialization.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_curried_toolzlike.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      copying cytoolz/tests/test_itertoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
      running build_ext
      building 'cytoolz.dicttoolz' extension
      creating build/temp.macosx-10.9-universal2-3.10
      creating build/temp.macosx-10.9-universal2-3.10/cytoolz
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/hristotodorov/Documents/Vyper/env/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c cytoolz/dicttoolz.c -o build/temp.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.o
      clang -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 -g build/temp.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.o -o build/lib.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.cpython-310-darwin.so
      building 'cytoolz.functoolz' extension
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/hristotodorov/Documents/Vyper/env/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c cytoolz/functoolz.c -o build/temp.macosx-10.9-universal2-3.10/cytoolz/functoolz.o
      cytoolz/functoolz.c:23087:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                  ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                        ^
      cytoolz/functoolz.c:23087:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                  ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                      ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cytoolz/functoolz.c:23092:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                  ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                        ^
      cytoolz/functoolz.c:23092:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                  ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                      ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cytoolz/functoolz.c:23176:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                  ret = _PyGen_Send((PyGenObject*)yf, NULL);
                        ^
      cytoolz/functoolz.c:23176:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                  ret = _PyGen_Send((PyGenObject*)yf, NULL);
                      ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      3 warnings and 3 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1
      ----------------------------------------
      ERROR: Failed building wheel for cytoolz
      Running setup.py clean for cytoolz
    Failed to build cytoolz
    Installing collected packages: cytoolz
        Running setup.py install for cytoolz ... error
        ERROR: Command errored out with exit status 1:
         command: /Users/hristotodorov/Documents/Vyper/env/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"'; __file__='"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-record-s_w__7qh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/hristotodorov/Documents/Vyper/env/include/site/python3.10/cytoolz
             cwd: /private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/
        Complete output (77 lines):
        ALERT: Cython not installed.  Building without Cython.
        running install
        /Users/hristotodorov/Documents/Vyper/env/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        running build
        running build_py
        creating build
        creating build/lib.macosx-10.9-universal2-3.10
        creating build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/compatibility.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/utils_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/_version.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/__init__.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/_signatures.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        creating build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
        copying cytoolz/curried/operator.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
        copying cytoolz/curried/__init__.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
        copying cytoolz/curried/exceptions.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/curried
        copying cytoolz/itertoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/dicttoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/functoolz.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/recipes.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/utils.pyx -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/utils.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/__init__.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/recipes.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/functoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/dicttoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/cpython.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        copying cytoolz/itertoolz.pxd -> build/lib.macosx-10.9-universal2-3.10/cytoolz
        creating build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_none_safe.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_utils.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_curried.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_compatibility.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_embedded_sigs.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_functoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_inspect_args.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_doctests.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_tlz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_signatures.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/dev_skip_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_recipes.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_docstrings.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_dev_skip_test.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_dicttoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_serialization.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_curried_toolzlike.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        copying cytoolz/tests/test_itertoolz.py -> build/lib.macosx-10.9-universal2-3.10/cytoolz/tests
        running build_ext
        building 'cytoolz.dicttoolz' extension
        creating build/temp.macosx-10.9-universal2-3.10
        creating build/temp.macosx-10.9-universal2-3.10/cytoolz
        clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/hristotodorov/Documents/Vyper/env/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c cytoolz/dicttoolz.c -o build/temp.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.o
        clang -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 -g build/temp.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.o -o build/lib.macosx-10.9-universal2-3.10/cytoolz/dicttoolz.cpython-310-darwin.so
        building 'cytoolz.functoolz' extension
        clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/hristotodorov/Documents/Vyper/env/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c cytoolz/functoolz.c -o build/temp.macosx-10.9-universal2-3.10/cytoolz/functoolz.o
        cytoolz/functoolz.c:23087:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                    ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                          ^
        cytoolz/functoolz.c:23087:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                    ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        cytoolz/functoolz.c:23092:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                    ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                          ^
        cytoolz/functoolz.c:23092:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                    ret = _PyGen_Send((PyGenObject*)yf, value == Py_None ? NULL : value);
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        cytoolz/functoolz.c:23176:19: error: implicit declaration of function '_PyGen_Send' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                    ret = _PyGen_Send((PyGenObject*)yf, NULL);
                          ^
        cytoolz/functoolz.c:23176:17: warning: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion]
                    ret = _PyGen_Send((PyGenObject*)yf, NULL);
                        ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        3 warnings and 3 errors generated.
        error: command '/usr/bin/clang' failed with exit code 1
        ----------------------------------------
    ERROR: Command errored out with exit status 1: /Users/hristotodorov/Documents/Vyper/env/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"'; __file__='"'"'/private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-install-0ebpvw2k/cytoolz_455f12fc06374667a15ffeeafa952f0f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/5y/q2vqps3s0jq7nc73z2zgmpmr0000gn/T/pip-record-s_w__7qh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/hristotodorov/Documents/Vyper/env/include/site/python3.10/cytoolz Check the logs for full command output.
    
    opened by HristoHT 15
  • Fails to build on Python 3.8

    Fails to build on Python 3.8

    Using Python 3.8, cytoolz cannot build/install.

    It appears to fail while compiling the dicttoolz.c file.

    See here for the full log: https://pastebin.com/hrYA77um

    If you have docker, you can quickly/easily replicate this with docker run python:3.8-rc python -m pip install cytoolz==0.10.0

    Note that this appears to be related (and would be resolved) by fixing #134, as I can confirm that cloning the repository and using the 0.10.0 tag, I can successfully build and install the package with python setup.py build --cython after removing all of the generated .c files on python 3.8.

    opened by cheeseandcereal 14
  • dicttoolz.pyx: faster iteration over dicts (10% speedup for keymap and friends)

    dicttoolz.pyx: faster iteration over dicts (10% speedup for keymap and friends)

    This speeds up several operations in dicttoolz.pyx by using PyDict_Next instead of a Cython loop over the iteritems. Using the example predicate for keyfilter, I got the following results. Before:

    >>> iseven = lambda x: x % 2 == 0
    >>> d = dict(enumerate(range(1, 25000)))
    >>> %timeit keyfilter(iseven, d)
    100 loops, best of 3: 3.19 ms per loop
    

    After:

    >>> %timeit keyfilter(iseven, d)
    100 loops, best of 3: 2.85 ms per loop
    

    I tried to apply the recipe to c_merge_with but found no difference in performance with large dicts, nor with a combination of large and small dicts, so I concluded that dict iteration isn't the bottleneck there (while the code became a lot more complicated). I did not try the many dicts use case.

    opened by larsmans 12
  • add conda.yaml and .binstar.yml files

    add conda.yaml and .binstar.yml files

    This provides hooks into an automated build process on binstar build.

    We should be able to set it up so that we can do something like the following

    conda install -c USERNAME cytoolz
    

    Where USERNAME is something like mrocklin or eriknw or pytoolz-dev. And it will download and install a binary on either Linux or OS-X, no C compiler required.

    opened by mrocklin 11
  • import failure with cython master

    import failure with cython master

    With cython master (and cpython master, but that seems to not matter here)

    Process Python finished
    Python 3.9.0a5+ (heads/master:71a3522ef8, Mar 26 2020, 23:28:49) 
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
    
    In [1]: import cytoolz
    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-2-fe6152314c51> in <module>
    ----> 1 import cytoolz
    
    ~/source/other_source/cytoolz/cytoolz/__init__.py in <module>
         22 from . import curried  # sandbox
         23 
    ---> 24 functoolz._sigs.update_signature_registry()
         25 
         26 from ._version import __version__, __toolz_version__
    
    AttributeError: partially initialized module 'cytoolz' has no attribute 'update_signature_registry' (most likely due to a circular import)
    
    In [3]: import cytoolz
    ---------------------------------------------------------------------------
    NameError                                 Traceback (most recent call last)
    <ipython-input-3-fe6152314c51> in <module>
    ----> 1 import cytoolz
    
    ~/.virtualenvs/bleeding/lib/python3.9/site-packages/cytoolz/__init__.py in <module>
         17 
         18 # Always-curried functions
    ---> 19 flip = functoolz.flip = curry(functoolz.flip)
         20 memoize = functoolz.memoize = curry(functoolz.memoize)
         21 
    
    NameError: name 'functoolz' is not defined
    

    This works correctly with the latest cython on pypi (0.29.16).

    I noticed this a month or two ago but only just now tracked down the source of the problem. I am unfortunately out of my depth to try and debug this.

    opened by tacaswell 10
  • Partition

    Partition

    OK, here is a "getting the feet wet" pull request.

    At first I just relied on zip and izip_longest, as we do in toolz. This yielded a 30% speedup.

    Then I tried making an iterator class and ran into weird issues. I get a fair number of stackoverflows. In particular I get some strange integer wraparound at 256. Does Cython use one byte integers somehow? Anyway, if you have any thoughts on debugging this I'd be interested.

    Fail case on my machine

    import cytoolz
    list(cytoolz.partition(5, range(1000)))
    

    oddly this works

    list(cytoolz.partition(5, range(200)))
    

    but this doesn't

    timeit list(cytoolz.partition(5, range(200)))
    
    opened by mrocklin 10
  • possible unitialized values in PyTupleObject

    possible unitialized values in PyTupleObject

    The code in sliding_window in itertoolz.pyx and here too can create a PyTupleObject with uninitialized items. I added some print(n) to the code, and the test_sliding_window code did not assign to self.prev[0]. If my guess is correct, this will cause PyPy to crash. All values must be assigned in creating a PyTupleObject. CPython will not check this, and will crash later if you ever access an uninitialized item, PyPy checks it and crashes earlier.

    opened by mattip 9
  • 'cytoolz.functoolz.Compose' object has no attribute '__module__'

    'cytoolz.functoolz.Compose' object has no attribute '__module__'

    When porting to cytoolz from toolz, I ran into the problem where the some previous library code was expecting Compose to have __module__ defined. I'm new to the project, but would expect the goal to be to generally to be able to move to cytoolz from toolz without any issues.

    Should this be defined in Compose? If so, I can work to add it.

    opened by djrtwo 9
  • Cython Version

    Cython Version

    I tried building with Cython and got the exception below. I'm pretty sure this is due to using a very old version of Cython as it seems to be something they fixed ( http://trac.cython.org/ticket/542 ).

    NotImplementedError("New relative imports.")
    

    It would be nice to know what version Cython cytoolz aims to be compatible with. Could we please get a note in the Readme? If I'm just missing it, I would appreciate a helpful pointer. Thanks in advance.

    opened by jakirkham 8
  • unified testing via `toolz`

    unified testing via `toolz`

    In the interest of development speed, the tests from toolz were basically just copied over to cytoolz. This is not easily sustainable. We would like all toolz-related tests to reside in toolz. Moreover, we would like a way to run the tests for toolz such as toolz.tests.run(), and to provide an optional toolz-like package on which to run the tests such as toolz.tests.run(cytoolz).

    This issue does not cover testing CyToolz C API, which I regard as very low priority at the moment.

    opened by eriknw 8
  • Bump pypa/cibuildwheel from 2.11.2 to 2.11.4

    Bump pypa/cibuildwheel from 2.11.2 to 2.11.4

    Bumps pypa/cibuildwheel from 2.11.2 to 2.11.4.

    Release notes

    Sourced from pypa/cibuildwheel's releases.

    v2.11.4

    • 🐛 Fix a bug that caused missing wheels on Windows when a test was skipped using CIBW_TEST_SKIP (#1377)
    • 🛠 Updates CPython 3.11 to 3.11.1 (#1371)
    • 🛠 Updates PyPy 3.7 to 3.7.10, except on macOS which remains on 7.3.9 due to a bug. (#1371)
    • 📚 Added a reference to abi3audit to the docs (#1347)

    v2.11.3

    • ✨ Improves the 'build options' log output that's printed at the start of each run (#1352)
    • ✨ Added a friendly error message to a common misconfiguration of the CIBW_TEST_COMMAND option - not specifying path using the {project} placeholder (#1336)
    • 🛠 The GitHub Action now uses Powershell on Windows to avoid occasional incompabilities with bash (#1346)
    Changelog

    Sourced from pypa/cibuildwheel's changelog.

    v2.11.4

    24 Dec 2022

    • 🐛 Fix a bug that caused missing wheels on Windows when a test was skipped using CIBW_TEST_SKIP (#1377)
    • 🛠 Updates CPython 3.11 to 3.11.1 (#1371)
    • 🛠 Updates PyPy to 7.3.10, except on macOS which remains on 7.3.9 due to a bug on that platform. (#1371)
    • 📚 Added a reference to abi3audit to the docs (#1347)

    v2.11.3

    5 Dec 2022

    • ✨ Improves the 'build options' log output that's printed at the start of each run (#1352)
    • ✨ Added a friendly error message to a common misconfiguration of the CIBW_TEST_COMMAND option - not specifying path using the {project} placeholder (#1336)
    • 🛠 The GitHub Action now uses Powershell on Windows to avoid occasional incompabilities with bash (#1346)
    Commits
    • 27fc88e Bump version: v2.11.4
    • a7e9ece Merge pull request #1371 from pypa/update-dependencies-pr
    • b9a3ed8 Update cibuildwheel/resources/build-platforms.toml
    • 3dcc2ff fix: not skipping the tests stops the copy (Windows ARM) (#1377)
    • 1c9ec76 Merge pull request #1378 from pypa/henryiii-patch-3
    • 22b433d Merge pull request #1379 from pypa/pre-commit-ci-update-config
    • 98fdf8c [pre-commit.ci] pre-commit autoupdate
    • cefc5a5 Update dependencies
    • e53253d ci: move to ubuntu 20
    • e9ecc65 [pre-commit.ci] pre-commit autoupdate (#1374)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Getting a circular dependency error

    Getting a circular dependency error

    Traceback (most recent call last):
      File "main.py", line 11, in <module>
        import config
      File "/Users/satwik/code/hop23/hop23-backend/config.py", line 6, in <module>
        import web3
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/web3/__init__.py", line 6, in <module>
        from eth_account import (
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/eth_account/__init__.py", line 1, in <module>
        from eth_account.account import (
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/eth_account/account.py", line 8, in <module>
        from cytoolz import (
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/cytoolz/__init__.py", line 1, in <module>
        from .itertoolz import *
      File "cytoolz/itertoolz.pyx", line 17, in init cytoolz.itertoolz
      File "cytoolz/utils.pyx", line 10, in init cytoolz.utils
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/toolz/__init__.py", line 19, in <module>
        from . import curried, sandbox
      File "/Users/satwik/code/hop23/hop23-backend/venv/lib/python3.8/site-packages/toolz/curried/__init__.py", line 81, in <module>
        partial = toolz.curry(toolz.partial)
    AttributeError: partially initialized module 'toolz' has no attribute 'partial' (most likely due to a circular import)
    
    opened by satwikkansal 0
  • error 'No such file or directory: 'build/lib.linux-x86_64-3.9/cytoolz/_version.py'

    error 'No such file or directory: 'build/lib.linux-x86_64-3.9/cytoolz/_version.py'

    Hello,

    I'm trying to create a simple android app using kivy and buildozer on Google Colab, but when i compile the app i am getting this error 'No such file or directory: 'build/lib.linux-x86_64-3.9/cytoolz/_version.py'. I need cytoolz module because i use web3.py, which need it.

    I use the last version of cytoolz (0.12.0) with the last toolz and setuptools modules.

    How can i resolve this error?

    opened by tonyo008 0
  • Simplify interleave.

    Simplify interleave.

    Simplified the implementation of interleave and made it slightly faster.

    seqs = [range(1000) for i in range(100)]
    %timeit list(interleave(seqs))
    

    old: 5.07 ms ± 240 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) new: 4.03 ms ± 45.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

    opened by groutr 0
  • Update property declarations.

    Update property declarations.

    Update property declarations from the special, deprecated style to the newer style supported by cython. https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#properties

    opened by groutr 0
Releases(0.12.1)
  • 0.12.1(Dec 23, 2022)

  • 0.12.0(Jul 11, 2022)

    • Update to toolz 0.12.0 (#170)
      • See toolz release notes: https://github.com/pytoolz/toolz/releases/tag/0.12.0
    • Build wheels automatically with cibuildwheel! (#167, #168, #169, #170)
    • Support newer Python versions (#165, don't use inspect.getargspec in tests)
    • Improve import times
    • Use versioneer.py to manage versions
      • __version__ and __toolz_version__ are no longer available from cytoolz._version--use cython.__toolz_version__ instead.

    This is the first release with automatic wheel building, versioneer for version management, and auto-upload to PyPI 🤞

    Source code(tar.gz)
    Source code(zip)
  • 0.11.2(Nov 6, 2021)

Owner
null
Simple, elegant, Pythonic functional programming.

Coconut Coconut (coconut-lang.org) is a variant of Python that adds on top of Python syntax new features for simple, elegant, Pythonic functional prog

Evan Hubinger 3.6k Jan 3, 2023
A fancy and practical functional tools

Funcy A collection of fancy functional tools focused on practicality. Inspired by clojure, underscore and my own abstractions. Keep reading to get an

Alexander Schepanovski 2.9k Dec 29, 2022
A functional standard library for Python.

Toolz A set of utility functions for iterators, functions, and dictionaries. See the PyToolz documentation at https://toolz.readthedocs.io LICENSE New

null 4.1k Jan 3, 2023
🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)

srsly: Modern high-performance serialization utilities for Python This package bundles some of the best Python serialization libraries into one standa

Explosion 329 Dec 28, 2022
A PyTorch implementation of a Factorization Machine module in cython.

fmpytorch A library for factorization machines in pytorch. A factorization machine is like a linear model, except multiplicative interaction terms bet

Jack Hessel 167 Jul 6, 2022
An alternative serializer implementation for REST framework written in cython built for speed.

drf-turbo An alternative serializer implementation for REST framework written in cython built for speed. Free software: MIT license Documentation: htt

Mng 74 Dec 30, 2022
Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web applications in Python 3.6 and above with performance auto-tuning. Optionally with Alpine Linux.

Supported tags and respective Dockerfile links python3.8, latest (Dockerfile) python3.7, (Dockerfile) python3.6 (Dockerfile) python3.8-slim (Dockerfil

Sebastián Ramírez 2.1k Dec 31, 2022
Just a small test with lists in cython

Test for lists in cython Algorithm create a list of 10^4 lists each with 10^4 floats values (namely: 0.1) - 2 nested for iterate each list and compute

Federico Simonetta 32 Jul 23, 2022
orfipy is a tool written in python/cython to extract ORFs in an extremely and fast and flexible manner

Introduction orfipy is a tool written in python/cython to extract ORFs in an extremely and fast and flexible manner. Other popular ORF searching tools

Urminder Singh 34 Nov 21, 2022
[Cython] Vs [Python] Which one is Faster ?

[Cython] Vs [Python] ? Attractive Contrast :) Mission : Which one is Faster ? Comparing of Execution runtime for [Selection_sort] with Time Complexity

baqer marani 1 Dec 5, 2021
Example for how to package a Python library based on Cython.

Cython sample module This project is an example of a module that can be built using Cython. It is an upgrade from a similar model developed by Arin Kh

Juan José García Ripoll 4 Aug 28, 2022
Scalene: a high-performance, high-precision CPU and memory profiler for Python

scalene: a high-performance CPU and memory profiler for Python by Emery Berger 中文版本 (Chinese version) About Scalene % pip install -U scalene Scalen

Emery Berger 138 Dec 30, 2022
Fcpy: A Python package for high performance, fast convergence and high precision numerical fractional calculus computing.

Fcpy: A Python package for high performance, fast convergence and high precision numerical fractional calculus computing.

SciFracX 1 Mar 23, 2022
A high performance implementation of HDBSCAN clustering. http://hdbscan.readthedocs.io/en/latest/

HDBSCAN Now a part of scikit-learn-contrib HDBSCAN - Hierarchical Density-Based Spatial Clustering of Applications with Noise. Performs DBSCAN over va

Leland McInnes 91 Dec 29, 2022
High performance implementation of Extreme Learning Machines (fast randomized neural networks).

High Performance toolbox for Extreme Learning Machines. Extreme learning machines (ELM) are a particular kind of Artificial Neural Networks, which sol

Anton Akusok 174 Dec 7, 2022
peace-performance (Rust) binding for python. To calculate star ratings and performance points for all osu! gamemodes

peace-performance-python Fast, To calculate star ratings and performance points for all osu! gamemodes peace-performance (Rust) binding for python bas

null 9 Sep 19, 2022
Functional programming in Python: implementation of missing features to enjoy FP

Fn.py: enjoy FP in Python Despite the fact that Python is not pure-functional programming language, it's multi-paradigm PL and it gives you enough fre

Oleksii Kachaiev 3.3k Jan 4, 2023
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021
A quick and dirty QT Statusbar implementation for grabbing GIFs from Tenor, since there is no offical or unofficial one I found. This was intended for use under Linux, however it was also functional enough on MacOS.

Statusbar-TenorGIF App for Linux A quick and dirty QT Statusbar implementation for grabbing GIFs from Tenor, since there is no offical one and I didnt

Luigi DaVinci 1 Nov 1, 2021
A functional and efficient python implementation of the 3D version of Maxwell's equations

py-maxwell-fdfd Solving Maxwell's equations via A python implementation of the 3D curl-curl E-field equations. This code contains additional work to e

Nathan Zhao 12 Dec 11, 2022