Don't really know does it fits into your library (I mean, you decide what is relevant), but when I wrote my own «restarter» I use some fault_result argument, which returned as function result after attempts gone. In my case this was made to avoid try/except/pass things, cause function fault was like «Oh, fault! Ok, will call it later anyway, does not matter…». It looks like right now we can't do that without external try/except/pass wrapper.

Hi, I'd like to use riprova.Retrier inside my custom class. e.g. I want to try to re-login every time say_trier fails or raises an exception. Is this sample code right? Didn't find a nice way to use decorater for say_trier.

class SayHello(object):
    def login(self):
        print "log in"
        do_login_process()

    def say(self):
        retrier = riprova.Retrier(backoff=riprova.ExponentialBackOff(interval=30), on_retry=self.on_retry)
        retrier.run(self.say_trier)

    def say_trier(self):
        something_may_fail_here()

    def on_retry(self, err, next_try):
        print('Operation error: {}'.format(err))
        print('Next try in: {}ms'.format(next_try))
        self.login()

sa = SayHello()
sa.say()

I see nothing wrong with using third party libraries, but in some cases it looks like is-number npm package :laughing:. It means, that js is poor, since it need such third-party «patches». I don't want to think, that python is poor, since it is not.

• paco.wraps should be removed cause it's enough to wrap functions with asyncio.coroutine without inspecting it.

>>> def sync(): print("sync")
... 
>>> async def async_(): print("async")
... 
>>> asyncio.get_event_loop().run_until_complete(asyncio.coroutine(sync)())
sync
>>> asyncio.get_event_loop().run_until_complete(asyncio.coroutine(async_)())
async
>>>

• paco.TimeoutLimit, async_timeout used when you have couple of things inside with block. If you have one coroutine, which you want to limit, you should use asyncio.wait_for and you don't need to split logic for running with or without timeout. If there is no timeout, then your timeout should be None.

I hope this two suggestions are strong enough to exclude paco from dependencies.

with riprova.Retrier(backoff=ConstantBackoff()) as retry:
   retry.run(run_failed_task, 'foo', bar=1)

async with riprova.AsyncRetrier(backoff=ConstantBackoff()) as retry:
   await retry.run(run_failed_task, 'foo', bar=1)

Currently, the context manager re-raises exception by its type with no args. So, there is no error message anymore. Besides it In some cases, args are mandatory and such behavior may lead to weird error messages. So, we must re-raise an existing exception instance instead of creating the new one.

riprova cannot retry when a method raise asyncio.TimeoutError, even when asyncio.TimeoutError is set in blacklist. The reason is that asyncio.TimeoutError is handle explicitly in https://github.com/h2non/riprova/blob/v0.2.5/riprova/async_retrier.py#L238-L240, thus it cannot trigger retry. Is this an intended behavior?

The following is sample code to reprocedure this issue.

import asyncio
import riprova

BLACKLIST = riprova.ErrorBlacklist([
    asyncio.TimeoutError,
])

@riprova.retry(backoff=riprova.ConstantBackoff(interval=1, retries=3), error_evaluator=BLACKLIST.isretry)
async def func():
    print('func')
    raise asyncio.TimeoutError()

asyncio.get_event_loop().run_until_complete(func())

The func will only run once, instead of three times.

six 1.11 (which has been available for half a year now) doesn't have any backwards incompatible changes that I know of and some other libraries (like CherryPy) depend on newer six which makes it difficult to use them together. Let's relax the version restriction to support such installations.

timeout = 200 while interval=100 causes the target function to run either twice or thrice (due to machine speed), which makes the following test inconsistent:

def test_retrier_run_max_timeout(MagicMock):
    iterable = (ValueError, NotImplementedError, RuntimeError, Exception)
    task = MagicMock(side_effect=iterable)

    retrier = Retrier(timeout=200, backoff=ConstantBackoff(interval=100))

    with pytest.raises(RetryTimeoutError):
        retrier.run(task, 2, 4, foo='bar')

    assert task.called
    assert task.call_count >= 1
    task.assert_called_with(2, 4, foo='bar')

    assert retrier.attempts >= 1
    assert isinstance(retrier.error, NotImplementedError)

Sometimes the raised error would be NotImplementedError, while at other times RunTimeError

I changed this test in my pull request but the core problem needs to be fixed.

 # Get delay before next retry
 delay = self.backoff.next()

 # If backoff is ready
 if delay == Backoff.STOP:
 return raise_from(MaxRetriesExceeded('max retries exceeded'), err)

While messing around with the code I checked err value and was surprised to see ImportError:

err = {ImportError} No module named stackless

The error was PyCharm related, maybe a similar error would be caused if I forgot to pip install requirements?

This code ignores all errors, even if it is caused internally and not by the function.

A possible solution is to blacklist certain errors, such as ImportError

>>> import riprova
>>> import time
>>> def my_sleep(*args, **kwargs): print("my sleep", args, kwargs); time.sleep(*args, **kwargs)
... 
>>> my_sleep(1)
my sleep (1,) {}
>>> def fail(): raise Exception
... 
>>> riprova.retry(backoff=riprova.ConstantBackoff(interval=2, retries=5), sleep_fn=my_sleep)(fail)()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    riprova.retry(backoff=riprova.ConstantBackoff(interval=2, retries=5), sleep_fn=my_sleep)(fail)()
  File "/home/poh/pro/py/riprova/riprova/retry.py", line 132, in wrapper
    return retry_runner(*args, **kw)
  File "/home/poh/pro/py/riprova/riprova/retrier.py", line 294, in run
    delay = self._get_delay()
  File "/home/poh/pro/py/riprova/riprova/retrier.py", line 246, in _get_delay
    self.error)
  File "<string>", line 2, in raise_from
riprova.exceptions.MaxRetriesExceeded: max retries exceeded
>>> 

The reason is identical names for keyword only arguments for decorator and wrapper: https://github.com/h2non/riprova/blob/master/riprova/retry.py#L24 https://github.com/h2non/riprova/blob/master/riprova/retry.py#L120

The API might look like:

def error_evaluator(err):
   return not isinstance(err, (MyCustomError, AnotherCustomError))

@riprova.retry(error_evaluator=error_evaluator)
def task():
   pass

Few deprecation warnings. FYI.

/usr/local/lib/python3.8/site-packages/riprova/async_retrier.py:141: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
self.on_retry = asyncio.coroutine(on_retry) if on_retry else None

/usr/local/lib/python3.8/site-packages/riprova/async_retrier.py:279: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
return (yield from asyncio.wait_for(

When calling asyncio.gather with functions decorated retry, the retries would finish earlier than expected.

import asyncio
import logging

import riprova


logging.basicConfig(
    format='[%(asctime)s] %(levelname)s - %(message)s',
    datefmt="%Y-%m-%d %H:%M:%S",
    level=logging.INFO
)
LOG = logging.getLogger()


def do_retry(val):
    LOG.info("do retry %s", val)
    return True


@riprova.retry(
    backoff=riprova.ConstantBackoff(retries=5),
    error_evaluator=do_retry,
)
async def do_stuff(name):
    LOG.info('calling %s', name)
    if name == 'bar':
        await asyncio.sleep(1.0)
    else:
        await asyncio.sleep(0.1)

    raise ValueError('value-error-{}'.format(name))


async def process():
    coros = [do_stuff('foo'), do_stuff('bar')]
    # coros = [do_stuff('foo')]
    await asyncio.gather(*coros, return_exceptions=True)


loop = asyncio.get_event_loop()
loop.run_until_complete(process())
loop.close()

results:

[2018-03-29 19:17:42] INFO - calling bar
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:42] INFO - do retry value-error-foo
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - do retry value-error-bar
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo

The expected result is that foo and bar should be both called 6x.