Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.

Overview

Retrying

Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.

The simplest use case is retrying a flaky function whenever an Exception occurs until a value is returned.

import random
from retrying import retry

@retry
def do_something_unreliable():
    if random.randint(0, 10) > 1:
        raise IOError("Broken sauce, everything is hosed!!!111one")
    else:
        return "Awesome sauce!"

print do_something_unreliable()

Features

  • Generic Decorator API
  • Specify stop condition (i.e. limit by number of attempts)
  • Specify wait condition (i.e. exponential backoff sleeping between attempts)
  • Customize retrying on Exceptions
  • Customize retrying on expected returned result

Installation

To install retrying, simply:

$ pip install retrying

Or, if you absolutely must:

$ easy_install retrying

But, you might regret that later.

Examples

As you saw above, the default behavior is to retry forever without waiting.

@retry
def never_give_up_never_surrender():
    print "Retry forever ignoring Exceptions, don't wait between retries"

Let's be a little less persistent and set some boundaries, such as the number of attempts before giving up.

@retry(stop_max_attempt_number=7)
def stop_after_7_attempts():
    print "Stopping after 7 attempts"

We don't have all day, so let's set a boundary for how long we should be retrying stuff.

@retry(stop_max_delay=10000)
def stop_after_10_s():
    print "Stopping after 10 seconds"

Most things don't like to be polled as fast as possible, so let's just wait 2 seconds between retries.

@retry(wait_fixed=2000)
def wait_2_s():
    print "Wait 2 second between retries"

Some things perform best with a bit of randomness injected.

@retry(wait_random_min=1000, wait_random_max=2000)
def wait_random_1_to_2_s():
    print "Randomly wait 1 to 2 seconds between retries"

Then again, it's hard to beat exponential backoff when retrying distributed services and other remote endpoints.

@retry(wait_exponential_multiplier=1000, wait_exponential_max=10000)
def wait_exponential_1000():
    print "Wait 2^x * 1000 milliseconds between each retry, up to 10 seconds, then 10 seconds afterwards"

We have a few options for dealing with retries that raise specific or general exceptions, as in the cases here.

def retry_if_io_error(exception):
    """Return True if we should retry (in this case when it's an IOError), False otherwise"""
    return isinstance(exception, IOError)

@retry(retry_on_exception=retry_if_io_error)
def might_io_error():
    print "Retry forever with no wait if an IOError occurs, raise any other errors"

@retry(retry_on_exception=retry_if_io_error, wrap_exception=True)
def only_raise_retry_error_when_not_io_error():
    print "Retry forever with no wait if an IOError occurs, raise any other errors wrapped in RetryError"

We can also use the result of the function to alter the behavior of retrying.

def retry_if_result_none(result):
    """Return True if we should retry (in this case when result is None), False otherwise"""
    return result is None

@retry(retry_on_result=retry_if_result_none)
def might_return_none():
    print "Retry forever ignoring Exceptions with no wait if return value is None"

Any combination of stop, wait, etc. is also supported to give you the freedom to mix and match.

Contribute

  1. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
  2. Fork the repository on GitHub to start making your changes to the master branch (or branch off of it).
  3. Write a test which shows that the bug was fixed or that the feature works as expected.
  4. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS.
Comments
  • Just depend on six

    Just depend on six

    Instead of copying six functions into retrying codebase why not just require the six module itself? This was if six updates then you automatically get this update as well...

    question 
    opened by harlowja 13
  • Support for marked wait

    Support for marked wait

    Often times retrying is done in a stepping manner whereupon the first N attempts wait X ms, the next M attempts wait for Y ms, etc. This patch implements support for such waiting by introducing the notion of 'wait markers'. Wait markers are simply tuples that define a set of attempts and their corresponding waits to go into effect.

    A handful of unit tests are also included.

    opened by bodenr 10
  • Retry on coroutines

    Retry on coroutines

    Hi,

    I'm a happy user of retrying, and i would like to use its features to retry coroutines. I wrote a PoC at https://github.com/novafloss/jenkins-epo/blob/master/jenkins_epo/utils.py#L34 , but i would like to contribute a more sane and integrated support.

    So here is my contribution :) It comes with one unit test which i hope is enough for code coverage.

    What do you think of this ? I would like to make it as consistent as possible with existing code and minimal.

    Regards, Étienne

    opened by bersace 9
  • Support retrying as context manager

    Support retrying as context manager

    Would be very helpful to support retrying as a context manager:

    Something along the lines of:

    def do_something_unreliable():
        if random.randint(0, 10) > 1:
            raise IOError("Broken sauce, everything is hosed!!!111one")
        else:
            return "Awesome sauce!"
    
    with retry(wait_fixed=2000):
        do_something_unreliable()
    
    opened by odedfos 7
  • Extend API to allow specifying the exception class

    Extend API to allow specifying the exception class

    Adds an on_exception argument to specify the class of exception to catch. This provides a cleaner API to catching only specific exception classes.

    Opening this in part as a POC and to start discussion. I think it is a much nicer API for this use case and also solves the issue of catching BaseException by default.

    See #17

    opened by d0ugal 7
  • Adding a back-off algorithm: Exponential Backoff And Jitter

    Adding a back-off algorithm: Exponential Backoff And Jitter

    As described in https://www.awsarchitectureblog.com/2015/03/backoff.html, this allows better distribution of competing clients across time during the retries.

    opened by RobinNil 4
  • Callbacks for before/after attempts

    Callbacks for before/after attempts

    Adds two additional parameters to the Retrying() constructor:

    _before_attempts
    _after_attempts
    

    Each of these should be a reference to a function that takes a single argument, attempt_number.

    _before_attempts will be called before each attempt is run; _after_attempts is run after any unsuccessful attempt.

    Handy for logging status messages, etc in between attempts.

    opened by cyrusd 3
  • Raising a `RetryError` on failure.

    Raising a `RetryError` on failure.

    Hi, I just found your library and it looks to do exactly what I was looking for. However when launching the code below, I am getting a RetryError. Is is the intended behavior?

    from retrying import retry, RetryError
    
    @retry(wait_fixed=100,
           stop_max_attempt_number=10,
           wrap_exception=False)
    def foo():
        raise ValueError()
    
    try:
        foo()
    except ValueError:
        print "ValueError"
    except RetryError as e:
        print "RetryError"
    

    Intuitively, I was expecting a ValueError. That would let me use the retry decorator without modifying the rest of the code.

    bug 
    opened by simondolle 3
  • require six >= 1.7

    require six >= 1.7

    retrying uses six.wraps, which requires at least version 1.7: https://bitbucket.org/gutworth/six/src/35f6d89f826f63486214e4746af028ccaa088901/six.py?at=1.7.0#cl-689

    opened by jobevers 2
  • python 2.6 / AttributeError: 'module' object has no attribute 'wraps'

    python 2.6 / AttributeError: 'module' object has no attribute 'wraps'

    hi, does someone have a clue on this error ? thanks

    $ python -V Python 2.6.6 $$ >>> six.version '1.6.1'

    $ python setup.py test running test running egg_info writing requirements to retrying.egg-info/requires.txt writing retrying.egg-info/PKG-INFO writing top-level names to retrying.egg-info/top_level.txt writing dependency_links to retrying.egg-info/dependency_links.txt reading manifest file 'retrying.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'retrying.egg-info/SOURCES.txt' running build_ext Traceback (most recent call last): File "setup.py", line 57, in setup(**settings) File "/usr/lib/python2.6/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib/python2.6/distutils/dist.py", line 975, in run_commands self.run_command(cmd) File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 137, in run self.with_project_on_sys_path(self.run_tests) File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 117, in with_project_on_sys_path func() File "/usr/lib/python2.6/dist-packages/setuptools/command/test.py", line 146, in run_tests testLoader = loader_class() File "/usr/lib/python2.6/unittest.py", line 816, in init self.parseArgs(argv) File "/usr/lib/python2.6/unittest.py", line 843, in parseArgs self.createTests() File "/usr/lib/python2.6/unittest.py", line 849, in createTests self.module) File "/usr/lib/python2.6/unittest.py", line 613, in loadTestsFromNames suites = [self.loadTestsFromName(name, module) for name in names] File "/usr/lib/python2.6/unittest.py", line 576, in loadTestsFromName module = import('.'.join(parts_copy)) File "/root/retrying-master/test_retrying.py", line 245, in @retry(wait_fixed=50, retry_on_result=retry_if_result_none) File "/root/retrying-master/retrying.py", line 47, in wrap @six.wraps(f) AttributeError: 'module' object has no attribute 'wraps'

    opened by nesies 2
  • Use functools/six.wraps (or other).

    Use functools/six.wraps (or other).

    Instead of having a decorator which loses the parents docstrings and name and other the decorator should be using functools.wraps (or six.wraps). Or even better it seems like you could just use https://pypi.python.org/pypi/wrapt which is likely better than the wraps() methods.

    enhancement 
    opened by harlowja 2
  • Transfer ownership to tenacity

    Transfer ownership to tenacity

    Thanks for this useful package and work over the years.

    Given that your priorities now lie elsewhere, would you be open to transferring ownership of the PyPI package name to tenacity?

    https://github.com/jd/tenacity/issues/356

    No work on your part and a compatibility shim will be added.

    opened by groodt 0
  • Add type hints

    Add type hints

    I use mypy and want to have type hints for retrying to fix this error:

    error: Skipping analyzing "retrying": module is installed, but missing library stubs or py.typed marker  [import]
    note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
    

    Current non-optimal solution is to ignore by adding the following lines to the mypy.ini config.

    [mypy-retrying.*]
    ignore_missing_imports = True
    
    opened by pietermarsman 0
  • Update retrying.py

    Update retrying.py

    Add print_exception attribute for exception printing for every attempt. For example

    ####Define functions (before_attempts, after_attempts)#### def print_attemt(attempt): print(f"Start Attempt # {attempt}") return attempt

    def print_attemt_after(attempt): print(f"Attempt # {attempt} was failed") return attempt

    Then call a func function with retry (we also can use retry as a decorator with func definition)

    retry(stop_max_attempt_number=3, print_exception=True, before_attempts=print_attemt, after_attempts=print_attemt_after)(func)(par)

    After func call we get the below

    Start Attempt # 1 [Errno 2] No such file or directory: 'F:\TEST\test.txt' Attempt # 1 was failed Start Attempt # 2 [Errno 2] No such file or directory: 'F:\TEST\test.txt' Attempt # 2 was failed Start Attempt # 3 [Errno 2] No such file or directory: 'F:\TEST\test.txt' Attempt # 3 was failed

    opened by Sergfalt 0
  • Creating a new release from master

    Creating a new release from master

    Hey guys, could you please create a new release (1.3.4?) from master? I really need the latests commits that are found in master.

    Thanks in advance :)

    opened by chenvsy 0
Owner
Ray Holder
Ray Holder
Repo created for the purpose of adding any kind of programs and projects

Programs and Project Repository A repository for adding programs and projects of any kind starting from beginners level to expert ones Contributing to

Unicorn Dev Community 3 Nov 2, 2022
Repo created for the purpose of adding any kind of programs and projects

Programs and Project Repository A repository for adding programs and projects of any kind starting from beginners level to expert ones Contributing to

Unicorn Dev Community 3 Nov 2, 2022
Retrying library for Python

Tenacity Tenacity is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just

Julien Danjou 4.3k Jan 2, 2023
General Purpose Python Library by Techman

General Purpose Python Library by Techman

Jack Hubbard 0 Feb 9, 2022
Nimbus - Open Source Cloud Computing Software - 100% Apache2 licensed

⚠️ The Nimbus infrastructure project is no longer under development. ⚠️ For more information, please read the news announcement. If you are interested

Nimbus 194 Jun 30, 2022
Sodium is a general purpose programming language which is instruction-oriented

Sodium is a general purpose programming language which is instruction-oriented (a new programming concept that we are developing and devising)

Satin Wuker 22 Jan 11, 2022
A general-purpose wallet generator, for supported coins only

2gen A general-purpose generator for keys. Designed for all cryptocurrencies supporting the Bitcoin format of keys and addresses. Functions To enable

Vlad Usatii 1 Jan 12, 2022
Fabric mod where anyone can PR anything, concerning or not. I'll merge everything as soon as it works.

Guess What Will Happen In This Fabric mod where anyone can PR anything, concerning or not (Unless it's too concerning). I'll merge everything as soon

anatom 65 Dec 25, 2022
Always fill your package requirements without the user having to do anything! Simple and easy!

WSL Should now work always-fill-reqs-python3 Always fill your package requirements without the user having to do anything! Simple and easy! Supported

Hashm 7 Jan 19, 2022
Would upload anything I do with/related to brainfuck

My Brainfu*k Repo Basically wanted to create something with Brainfu*k but realized that with the smol brain I have, I need to see the cell values real

Rafeed 1 Mar 22, 2022
A python script to simplify recompiling, signing and installing reverse engineered android apps.

urszi.py A python script to simplify the Uninstall Recompile Sign Zipalign Install cycle when reverse engineering Android applications. It checks if d

Ahmed Harmouche 4 Jun 24, 2022
Python client SDK designed to simplify integrations by automating key generation and certificate enrollment using Venafi machine identity services.

This open source project is community-supported. To report a problem or share an idea, use Issues; and if you have a suggestion for fixing the issue,

Venafi, Inc. 13 Sep 27, 2022
solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

William Wolf 12 Jul 13, 2022
Project aims to map out common user behavior on the computer

User-Behavior-Mapping-Tool Project aims to map out common user behavior on the computer. Most of the code is based on the research by kacos2000 found

trustedsec 136 Dec 23, 2022
YunoHost is an operating system aiming to simplify as much as possible the administration of a server.

YunoHost is an operating system aiming to simplify as much as possible the administration of a server. This repository corresponds to the core code, written mostly in Python and Bash.

YunoHost 1.5k Jan 9, 2023
fetchmesh is a tool to simplify working with Atlas anchoring mesh measurements

A Python library for working with the RIPE Atlas anchoring mesh. fetchmesh is a tool to simplify working with Atlas anchoring mesh measurements. It ca

null 2 Aug 30, 2022
New multi tool im making adding features currently

Emera Multi Tool New multi tool im making adding features currently Current List of Planned Features - Linkvertise Bypasser - Discord Auto Bump - Gith

Lamp 3 Dec 3, 2021
Starscape is a Blender add-on for adding stars to the background of a scene.

Starscape Starscape is a Blender add-on for adding stars to the background of a scene. Features The add-on provides the following features: Procedural

Marco Rossini 5 Jun 24, 2022
A C-like hardware description language (HDL) adding high level synthesis(HLS)-like automatic pipelining as a language construct/compiler feature.

██████╗ ██╗██████╗ ███████╗██╗ ██╗███╗ ██╗███████╗ ██████╗ ██╔══██╗██║██╔══██╗██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝ ██████╔╝██║██████╔╝█

Julian Kemmerer 391 Jan 1, 2023