decorator

Overview

Decorators for Humans

The goal of the decorator module is to make it easy to define signature-preserving function decorators and decorator factories. It also includes an implementation of multiple dispatch and other niceties (please check the docs). It is released under a two-clauses BSD license, i.e. basically you can do whatever you want with it but I am not responsible.

Installation

If you are lazy, just perform

$ pip install decorator

which will install just the module on your system.

If you prefer to install the full distribution from source, including the documentation, clone the GitHub repo or download the tarball, unpack it and run

$ pip install .

in the main directory, possibly as superuser.

Testing

If you have the source code installation you can run the tests with

$ python src/tests/test.py -v

or (if you have setuptools installed)

$ python setup.py test

Notice that you may run into trouble if in your system there is an older version of the decorator module; in such a case remove the old version. It is safe even to copy the module decorator.py over an existing one, since we kept backward-compatibility for a long time.

Repository

The project is hosted on GitHub. You can look at the source here:

https://github.com/micheles/decorator

Documentation

The documentation has been moved to https://github.com/micheles/decorator/blob/master/docs/documentation.md

From there you can get a PDF version by simply using the print functionality of your browser.

Here is the documentation for previous versions of the module:

https://github.com/micheles/decorator/blob/4.3.2/docs/tests.documentation.rst https://github.com/micheles/decorator/blob/4.2.1/docs/tests.documentation.rst https://github.com/micheles/decorator/blob/4.1.2/docs/tests.documentation.rst https://github.com/micheles/decorator/blob/4.0.0/documentation.rst https://github.com/micheles/decorator/blob/3.4.2/documentation.rst

For the impatient

Here is an example of how to define a family of decorators tracing slow operations:

from decorator import decorator

@decorator
def warn_slow(func, timelimit=60, *args, **kw):
    t0 = time.time()
    result = func(*args, **kw)
    dt = time.time() - t0
    if dt > timelimit:
        logging.warn('%s took %d seconds', func.__name__, dt)
    else:
        logging.info('%s took %d seconds', func.__name__, dt)
    return result

@warn_slow  # warn if it takes more than 1 minute
def preprocess_input_files(inputdir, tempdir):
    ...

@warn_slow(timelimit=600)  # warn if it takes more than 10 minutes
def run_calculation(tempdir, outdir):
    ...

Enjoy!

Comments
  • Python 2.7 installation is broken

    Python 2.7 installation is broken

    Since you dropped python 2 support, you have to stop making universal wheels for this module, otherwise pypi is offering wrong version for python2 setups. I see that this was addressed in commit 50fa96bf370146602b02ece8378cdafa9f57684f

    Please push new version to pypi, otherwise we are having issues with installation on python2.

    opened by mast 16
  • Adding patch_argspec

    Adding patch_argspec

    I have a task where I need to perform complex decoration with patching function spec. But this library is still very useful in my case and allow to not reimplement a lot of decoration logic.

    wontfix 
    opened by Kontakter 12
  • args tuple error in v5

    args tuple error in v5

    I'm not sure if this is by design, but I have a lot of decorators that look like this

    @decorator
    def wrap(func_, *args, **kwargs):
        if not args[-1]:
            new_args = list(args)
            new_args[-1] = ENVIRONMENT_DEPENDENT_VAR
            args = tuple(new_args)
        return func_(*args, **kwargs)
    
    @wrap
    def do_stuff(requiredvar, optionalvar=None):
        pass # real work
        return 
    
    

    basically checking if a certain variable is set and, if not, setting it to some appropriate veriable depending on whatever situation.

    In v4, when calling do_stuff(some_var)

    the args value within wrap would be (some_var, None)

    in v5, the args value within wrap is now (some_var, )

    Is this intentional to no longer supply a None (or whatever the function's default is) for named positional arguments?

    opened by bauman 11
  • Output coroutine functions

    Output coroutine functions

    It would be great if @decorator could be used to create decorators which produce coroutine functions. Is there already such functionality and I just didn't find it? If not, would this be feasible? If so, are there any plans for this? Would you accept PRs?

    Example:

    from inspect import iscoroutinefunction
    from decorator import decorator
    
    @decorator
    async def ensure_coro(function, *args, **kwargs): 
        ret = function(*args, **kwargs)
        if iscoroutinefunction(function):
            ret = await ret
        return ret
    
    @ensure_coro
    def foo():
        return 42
    
    assert iscoroutinefunction(foo)
    
    opened by Victor-Savu 11
  • Python 3.6 - Trace example - kw is empty

    Python 3.6 - Trace example - kw is empty

    It looks like the kw array is not being preserved or passed at all in 3.6. I have not tested in previous versions, but based on the documentation it is a different behavior.

    Python 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from decorator import decorator
    >>> @decorator
    ... def trace(f, *args, **kw):
    ...     kwstr = ', '.join('%r: %r' % (k, kw[k]) for k in sorted(kw))
    ...     print("calling %s with args %s, {%s}" % (f.__name__, args, kwstr))
    ...
    >>> @trace
    ... def samplefn(x, a="a1"):
    ...     pass
    ...
    >>> samplefn(1)
    calling samplefn with args (1, 'a1'), {}
    >>> samplefn(1, "inline")
    calling samplefn with args (1, 'inline'), {}
    >>> samplefn(1, a="askeyword") # notice kwstr is coming back blank
    calling samplefn with args (1, 'askeyword'), {}
    
    opened by kdeenanauth 9
  • Looks like it works on staticmethods  in python3.7

    Looks like it works on staticmethods in python3.7

    In Python 3.5, many such limitations have been removed, but I still think that it is cleaner and safer to decorate only functions. If you want to decorate things like classmethods/staticmethods and general callables - which I will never support in the decorator module - I suggest you to look at the wrapt project by Graeme Dumpleton.

    from decorator import decorator
    import asyncio
    import time
    
    
    @decorator
    async def log_start_stop(func, *args, **kwargs):
        t0 = int(time.time())
        print(t0, f'core={func}, args={args},kwargs={kwargs}')
        result = await func(*args, **kwargs)
        t1 = int(time.time())
        print(t1, f'result={result}')
    
    
    class A:
        @staticmethod
        @log_start_stop
        async def make_task(n):
            for i in range(n):
                await asyncio.sleep(1)
            return 100
    
    
    a = A()
    print(a)
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(A.make_task(5))
    finally:
        loop.run_until_complete(loop.shutdown_asyncgens())
        loop.close()
    
    
    opened by yjqiang 8
  • New option in `decorator` to prepend positional arguments

    New option in `decorator` to prepend positional arguments

    This still needs a review and possibly improvements (such as also removing arguments by name and adding tests). Nevertheless it shows how it could be done.

    Fixes #55

    opened by smarie 8
  • Fully signature preserving equivalent of functools.wraps

    Fully signature preserving equivalent of functools.wraps

    On Python 3, functools.wraps nominally preserves function signatures, e.g.,

    import functools
    
    def original(x):
        return x
    
    @functools.wraps(original)
    def wrapper(*args, **kwargs):
        return original(*args, **kwargs)
    

    The function has the right associated signature object (so introspection works properly) but abstraction is still a little leaky, e.g., for handling invalid function arguments:

    In [11]: wrapper(y=1)
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-11-1e0b778ae20b> in <module>()
    ----> 1 wrapper(y=1)
    
    <ipython-input-4-878d38897098> in wrapper(*args, **kwargs)
          6 @functools.wraps(original)
          7 def wrapper(*args, **kwargs):
    ----> 8     return original(*args, **kwargs)
    
    TypeError: original() got an unexpected keyword argument 'y'
    

    I think the decorator library can produce fully compatible function signatures, but the interface is a little different:

    @functools.partial(decorate, original)
    def wrapper(original, *args, **kwargs):
        return original(*args, **kwargs)
    

    In particular, it's awkward that I need to include the original function in the wrapper signature (e.g., def wrapper(original, *args, **kwargs) rather than def wrapper(*args, **kwargs) .

    Is there an easy way to write a drop-in replacement for functools.wraps with the decorator library?

    opened by shoyer 8
  • kwargs inside the decorator are wrong

    kwargs inside the decorator are wrong

    Hello,

    I ran into a issue when I try to assess certain kwargs inside the decorator. In my situation I had one kwarg passed to the function, but it was inside args inside the decorator. Am I doing something wrong or is this a bug or is this a feature?

    The test below fails for me and prints args (True,) kwargs {}

    def test_decorator_kwargs():
        @decorator.decorator
        def loader(func, *args, **kwargs):
            print(f"args {args} kwargs {kwargs}")
            return kwargs
    
        @loader
        def my_loader(test=False):
            return test
    
        kwargs = my_loader(test=True)
        assert "test" in kwargs
        assert kwargs["test"]
    
        kwargs = my_loader(test=False)
        assert "test" in kwargs
        assert not kwargs["test"]
    

    I'm using decorator 4.4.2 on python 3.7.1

    opened by mxposed 7
  • Python 2 Deprecation Plan

    Python 2 Deprecation Plan

    Hey, I'm working on deleting Python 2 code in numpy/scipy, which includes a copy of decorator (yay!).

    I see you mention full backwards support to python 2.6 in src/tests/documentation.py, but that seems to have been written ~5 years ago. With Python 2 no longer officially supported would you be interested in dropping Python 2 support? It looks like it's possible it would allow for the clean up of a few pieces of logic. I'm happy to help out if so.

    opened by sethtroisi 7
  • Functionmaker.create (from doc example) fails with kwarg-only functions

    Functionmaker.create (from doc example) fails with kwarg-only functions

    Possibly related to #138 , but I'm creating a new issue here in case the root cause is fundamentally different.

    I'm using the third-party decorator example like so:

    In [1]: def foo(a, *, b, c=0):
       ...:     return a + b * c
       ...: 
    
    In [2]: from decorator import FunctionMaker
    
    In [3]: def decorator_apply(dec, func):
       ...:      """
       ...:      Decorate a function by preserving the signature even if dec
       ...:      is not a signature-preserving decorator.
       ...:      """
       ...:      return FunctionMaker.create(
       ...:          func, 'return decfunc(%(signature)s)',
       ...:          dict(decfunc=dec(func)), __wrapped__=func)
       ...: 
    
    In [4]: def dec(f):
       ...:     print(f)
       ...:     return f
    
    In [5]: bar = decorator_apply(dec, foo)
    <function foo at 0x7f520b0f5f70>
    Error in generated code:
    def foo(a, *, b=None, c=None):
        return decfunc(a, *, b=None, c=None)
    
    ╭──────────────────────────────────────────────────────────────────────────────╮
    │  <decorator-gen-0>:2                                                         │
    │     return decfunc(a, *, b=None, c=None)                                     │
    │                        ▲                                                     │
    ╰──────────────────────────────────────────────────────────────────────────────╯
    SyntaxError: iterable argument unpacking follows keyword argument unpacking
    
    

    I'm on decorator==5.1.0, python 3.9.9.

    Is there a recommended workaround here? Or is there something more fundamental that will prevent this from working?

    opened by bmcfee 6
  • decorator whether to support decorator partial functions?

    decorator whether to support decorator partial functions?

    I looked at the code and it wasn't supported. Can we add a judgment here that supports partial functions? src/decorator.py:74 f inspect.isroutine(func):

    opened by dafu-wu 0
  • Not preserving low level signature metadada

    Not preserving low level signature metadada

    (Sorry if the title is somewhat misleading or I did get some jargon wrong)

    In version 5.1.1 methods decorated by decorate created with the decorator lib stopped working with qt(pyqt?) signals that inspect the signature of the connected slot (also tested with 4.0.2 with a slight different error).

    Failing snippet:

    @decorator
    def dummy_decorator(f, *args, **kwargs):
        return f(*args, **kwargs)
    
     class Foo:
        def __init__(self):
            self.action = QAction()
            self.action.triggered.connect(self.ActionCallbackNoArg)
        @dummy_decorator
        def ActionCallbackNoArg(self):
            pass
    
    foo = Foo()
    foo.action.trigger()
    

    This was not an issue in 4.4.2. While porting our code base to python 3.10 the decorator version also got upgraded, decorator version prior to this was 4.4.2. We will probably be able to downgrade back for now.

    I created a test to reproduce the error. add_qt_test_patch.txt

    Create env and install decorator in editable mode:

    conda create -n decorator python=3.6 pyqt
    conda activate decorator
    pip install --editable .
    

    Test with 4.4.2 (pass):

    W:\my\Projects\decorator (master -> origin)
    (decorator) λ git checkout -- src/tests/test.py
    
    W:\my\Projects\decorator (master -> origin)
    (decorator) λ git checkout 4.4.2
    Note: switching to '4.4.2'.
    ...
    HEAD is now at 55a68b5 Updated setup.py [skip CI]
    
    W:\my\Projects\decorator (HEAD detached at 55a68b5)
    (decorator) λ patch -i add_qt_test_patch.txt -p 1
    patching file src/tests/test.py
    Hunk #1 succeeded at 18 with fuzz 1 (offset 6 lines).
    
    W:\my\Projects\decorator (HEAD detached at 55a68b5)
    (decorator) λ python src/tests/test.py -v QtActionTestCase.test_qt_decorator_signature_preserving_interaction_methods
    test_qt_decorator_signature_preserving_interaction_methods (__main__.QtActionTestCase) ... ok
    
    ----------------------------------------------------------------------
    Ran 1 test in 0.001s
    
    OK
    

    Test with 5.1.1 (fail, at the time of this writing origin/master will also fail):

    W:\my\Projects\decorator (HEAD detached at 55a68b5)
    (decorator) λ git checkout -- src/tests/test.py
    
    W:\my\Projects\decorator (HEAD detached at 55a68b5)
    (decorator) λ git checkout 5.1.1
    Previous HEAD position was 55a68b5 Updated setup.py [skip CI]
    HEAD is now at ad013a2 Updated changelog and bumped version to 5.1.1
    
    W:\my\Projects\decorator (HEAD detached at ad013a2)
    (decorator) λ patch -i add_qt_test_patch.txt -p 1
    patching file src/tests/test.py
    
    W:\my\Projects\decorator (HEAD detached at ad013a2)
    (decorator) λ python src/tests/test.py -v QtActionTestCase.test_qt_decorator_signature_preserving_interaction_methods
    test_qt_decorator_signature_preserving_interaction_methods (__main__.QtActionTestCase) ... Traceback (most recent call last):
      File "w:\my\projects\decorator\src\decorator.py", line 231, in fun
        args, kw = fix(args, kw, sig)
      File "w:\my\projects\decorator\src\decorator.py", line 203, in fix
        ba = sig.bind(*args, **kwargs)
      File "W:\my\envs\decorator\lib\inspect.py", line 2997, in bind
        return args[0]._bind(args[1:], kwargs)
      File "W:\my\envs\decorator\lib\inspect.py", line 2918, in _bind
        raise TypeError('too many positional arguments') from None
    TypeError: too many positional arguments
    

    I also was somewhat curious. "Plain" functions:

    >>> plain.ActionCallbackNoArg.__func__.__code__.co_argcount
    1
    >>> plain.ActionCallbackNoArg.__func__.__code__.co_varnames
    ('self',)
    >>> plain.ActionCallbackAnyArgs.__func__.__code__.co_argcount
    1
    >>> plain.ActionCallbackAnyArgs.__func__.__code__.co_varnames
    ('self', 'args', 'kwargs')
    

    Decorated with 4.4.2:

    >>> decorated.ActionCallbackNoArg.__func__.__code__.co_argcount
    1
    >>> decorated.ActionCallbackNoArg.__func__.__code__.co_varnames
    ('self',)
    >>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_argcount
    1
    >>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_varnames
    ('self', 'args', 'kwargs')
    

    Decorated with 5.1.1:

    >>> decorated.ActionCallbackNoArg.__func__.__code__.co_argcount
    0
    >>> decorated.ActionCallbackNoArg.__func__.__code__.co_varnames
    ('args', 'kw')
    >>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_argcount
    0
    >>> decorated.ActionCallbackAnyArgs.__func__.__code__.co_varnames
    ('args', 'kw')
    
    opened by prusse-martin 1
  • Continuous fuzzing by way of OSS-Fuzz

    Continuous fuzzing by way of OSS-Fuzz

    Hi,

    I was wondering if you would like to integrate continuous fuzzing by way of OSS-Fuzz? Fuzzing is a way to automate test-case generation and has been heavily used for memory unsafe languages. Recently efforts have been put into fuzzing memory safe languages and Python is one of the languages that has recently gotten better tool support.

    In this https://github.com/google/oss-fuzz/pull/7998 we did an initial integration into OSS-Fuzz. OSS-Fuzz is a free service run by Google that performs continuous fuzzing of important open source projects. If you would like to integrate, the only thing I need is a list of email(s) that will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats. Notice the emails affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

    opened by DavidKorczynski 0
  • `FunctionMaker.create` raises unexpected SyntaxError when return is present as substring for async function

    `FunctionMaker.create` raises unexpected SyntaxError when return is present as substring for async function

    About

    In current FunctionMaker.create, if the function to create from is an async function and if the function body contains any occurrence of return, it will blindly replace it with return await. This will raise a SyntaxError in edge cases where return is present not as the keyword return but a substring.

    Reproduce

    from decorator import FunctionMaker
    
    
    async def IDENTITY(return_value):
        return return_value
    
    FunctionMaker.create('WILL_RAISE_SYNTAX_ERROR()', 'return IDENTITY(return_value=actual_return_value)', dict(IDENTITY=IDENTITY, _call_=IDENTITY, actual_return_value=10), addsource=True)
    

    Error Message

    Error in generated code:
    async def WILL_RAISE_SYNTAX_ERROR():
        return await IDENTITY(return await_value=actual_return await_value)
    
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/opt/conda/lib/python3.9/site-packages/decorator.py", line 196, in create
        return self.make(body, evaldict, addsource, **attrs)
      File "/opt/conda/lib/python3.9/site-packages/decorator.py", line 159, in make
        code = compile(src, filename, 'single')
      File "<decorator-gen-0>", line 2
        return await IDENTITY(return await_value=actual_return await_value)
                              ^
    SyntaxError: invalid syntax
    

    Analysis

    FunctionMaker.create has the following:

    if caller and iscoroutinefunction(caller):
        body = ('async def %(name)s(%(signature)s):\n' + ibody).replace(
            'return', 'return await')
    

    However, this replacement is blind, if a parameter is named like return_value, it will be also replaced as it contains return (like the above example).

    Moreover, the following lines can produce similar errors:

    • return_value = ... A simple assignment statement
    • "This is a long message containing a return word"

    Fix

    To fix, we need to match return as a keyword rather than as any substring. A straightforward fix looks like the following (not fully tested):

    if caller and iscoroutinefunction(caller):
        rawbody = "async def %(name)s(%(signature)s):\n" + ibody
        body = re.sub(r"(^|\s)return(\s)", r"\1return await\2", rawbody)
    
    opened by pengzhengyi 0
  • `FunctionMaker.create()` fails when given a function definition with a return type annotation.

    `FunctionMaker.create()` fails when given a function definition with a return type annotation.

    I think I may have found an issue with FunctionMaker.create(). When you try to pass a string containing a return type annotation, it breaks this function. Consider this example:

    from decorator import FunctionMaker
    
    def add(a: int, b: int) -> int:
        return a + b
    
    add2 = FunctionMaker.create("add2(a: int, b: int)", "return add(a, b)", evaldict={"add": add}) # No problem
    assert add2(6, 8) == add(6, 8) # add2 does the same as add
    
    add3 = FunctionMaker.create("add3(a: int, b: int) -> int", "return add(a, b)", evaldict={"add": add}) # ERROR
    

    I get a message like this:

      File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3457, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
    
      File "<ipython-input-9-5d5ad6515055>", line 9, in <module>
        add3 = FunctionMaker.create("add3(a: int, b: int) -> int", "return add(a, b)", evaldict={"add": add})
    
      File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/decorator.py", line 222, in create
        return self.make(body, evaldict, addsource, **attrs)
    
      File "/nfs/iil/proj/dt/prv08/airon/git/socb_client/venv/lib/python3.9/site-packages/decorator.py", line 184, in make
        code = compile(src, filename, 'single')
    
      File "<decorator-gen-123>", line 1
        def add3(a: int, b: int) -> in):
                                    ^
    SyntaxError: invalid syntax
    

    It seems like the code that decomposes the signature at https://github.com/micheles/decorator/blob/9cf8151de8fd26551cb43288b2010965ad346839/src/decorator.py#L181

    assumes the end of the signature is always a right parenthesis, which is not the case when we specify a return type annotation.

    Then, the code at https://github.com/micheles/decorator/blob/9cf8151de8fd26551cb43288b2010965ad346839/src/decorator.py#L195 unconditionally adds the right parenthesis that was (actually or supposedly) removed previously.

    I could fix this issue in my own environment, but the fix is kind of dirty, and has not gone through all your testing, obviously.

    opened by amitaiirron 1
Releases(4.4.2)
Owner
Michele Simionato
Michele Simionato
Python decorator for `TODO`s

Python decorator for `TODO`s. Don't let your TODOs rot in your python projects anymore !

Klemen Sever 74 Sep 13, 2022
This package tries to emulate the behaviour of syntax proposed in PEP 671 via a decorator

Late-Bound Arguments This package tries to emulate the behaviour of syntax proposed in PEP 671 via a decorator. Usage Mention the names of the argumen

Shakya Majumdar 0 Feb 6, 2022
decorator

Decorators for Humans The goal of the decorator module is to make it easy to define signature-preserving function decorators and decorator factories.

Michele Simionato 734 Dec 30, 2022
API Rate Limit Decorator

ratelimit APIs are a very common way to interact with web services. As the need to consume data grows, so does the number of API calls necessary to re

Tomas Basham 574 Dec 26, 2022
Decorator for PyMC3

sampled Decorator for reusable models in PyMC3 Provides syntactic sugar for reusable models with PyMC3. This lets you separate creating a generative m

Colin 50 Oct 8, 2021
API Rate Limit Decorator

ratelimit APIs are a very common way to interact with web services. As the need to consume data grows, so does the number of API calls necessary to re

Tomas Basham 575 Jan 5, 2023
esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch.

esguard esguard provides a Python decorator that waits for processing while monitoring the load of Elasticsearch. Quick Start You need to launch elast

po3rin 5 Dec 8, 2021
Python decorator for `TODO`s

Python decorator for `TODO`s. Don't let your TODOs rot in your python projects anymore !

Klemen Sever 74 Sep 13, 2022
Python @deprecat decorator to deprecate old python classes, functions or methods.

deprecat Decorator Python @deprecat decorator to deprecate old python classes, functions or methods. Installation pip install deprecat Usage To use th

null 12 Dec 12, 2022
This package tries to emulate the behaviour of syntax proposed in PEP 671 via a decorator

Late-Bound Arguments This package tries to emulate the behaviour of syntax proposed in PEP 671 via a decorator. Usage Mention the names of the argumen

Shakya Majumdar 0 Feb 6, 2022
A Runtime method overload decorator which should behave like a compiled language

strongtyping-pyoverload A Runtime method overload decorator which should behave like a compiled language there is a override decorator from typing whi

null 20 Oct 31, 2022