Exemplo de implementação do padrão circuit breaker em python

Overview

fast-circuit-breaker

Circuit breakers existem para permitir que uma parte do seu sistema falhe sem destruir todo seu ecossistema de serviços. Michael Nygard

Nesse exemplo vamos executar o serviço de oferta (fria) que se comunica com o serviço de oferta do parceiro (quente). Depois vamos provocar uma indisponibilidade no serviço de oferta do parceiro, retornando uma oferta fria (fallback) do serviço de oferta.

Fluxo de oferta!

Veremos que em certo momento o serviço de oferta deixará de se comunicar com o serviço de oferta do parceiro, abrindo o circuito (open), após um determinado tempo o serviço de oferta continuará tentando restabelecer a comunicação com serviço de oferta do parceiro, circuito meio-aberto (half-open).

Quando a comunicação entre os serviços for restabelecida, o circuito será fechado (close).

Observe abaixo o fluxo de mudança de estado do padrão circuit breaker.

Estados do circuit breaker!

Instalação

Crie um ambiente virtual.

python3 -m venv venv

Ative o ambiente virtual.

source venv/bin/activate

Instale as dependências do projeto.

pip install -r requirements.txt

Uso

Execute o serviço de oferta do parceiro, responsável por retornar uma oferta quente (hot).

python partner_offer_service.py

Execute o serviço de oferta responsável por buscar oferta quente no serviço de oferta do parceiro.

HTTPX_LOG_LEVEL=debug python offer_service.py

Vamos testar a busca de oferta, através de uma chamada HTTP do qualquer cliente (browser, curl, httpie), o exemplo abaixo usa o httpie.

http ":8001/offer"

A resposta deve ser uma oferta quente do serviço de oferta do parceiro.

"Hot offer 24:48"

Veja nos logs do serviço de oferta, a resposta OK do serviço de oferta do parceiro.

DEBUG [2021-06-19 11:03:03] httpx._client - HTTP Request: GET http://127.0.0.1:8000/offer/hot "HTTP/1.1 200 OK"

Circuit breaker

Vamos alterar o arquivo partner_offer_service.py na linha 13 para retornar o código de erro 500 na resposta do recurso GET /offer/hot, conforme exemplo abaixo.

return Response(content=body, status_code=500)

Atenção: os serviços tem a configuração de recarregar (reload) a aplicação toda vez que um arquivo é alterado.

Vamos chamar o serviço de busca de oferta novamente.

http ":8001/offer"

A resposta agora deve ser uma oferta fria, retornada através de uma função (fallback) do serviço de oferta.

"Cold offer fallback 47:32"

Veja nos logs do serviço de oferta um erro na comunicação com o serviço de oferta do parceiro.

DEBUG [2021-06-19 20:44:27] httpx._client - HTTP Request: GET http://127.0.0.1:8000/offer/hot "HTTP/1.1 500 Internal Server Error"

Vamos verificar o estado do circuito do serviço de oferta.

http ":8001/offer/circuit"

A resposta mostra que o circuito está com o estado fechado (current_state) e 1 falha fail_counter.

{
  "current_state": "closed",
  "fail_counter": 1
}

Antes de prosseguirmos vamos analisar a configuração do circuito no arquivo circuit_breaker.py, para mais informações consulte a documentação da biblioteca pybreaker.

  1. fail_max: Quantidade máxima de falhas.
  2. reset_timeout: Limite de tempo (segundos) para redefinição do estado do circuito.
  3. state_storage: Onde o estado será armazenado (Memória, Redis, etc).
  4. listeners: Ouvintes que serão notificados em cada evento do circuito
circuit_breaker = CircuitBreaker(
    fail_max=3,
    reset_timeout=15,
    state_storage=state_storage,
    listeners=[LogListener()]
)

Vamos chamar o recurso de buscar oferta mais 3 vezes.

http ":8001/offer"

Após 3 falhas (fail_max) na comunicação com o serviço de oferta do parceiro, o circuito é aberto (open).

Vamos verificar o estado do circuito mais uma vez.

http ":8001/offer/circuit"

Na resposta o circuito está aberto (current_state) com 3 falhas fail_counter.

{
  "current_state": "open",
  "fail_counter": 3
}

Observe que no estado aberto, não há registro de log de comunicação, pois o circuito protege o serviço de oferta do parceiro de receber chamadas por um determinado período de tempo.

No estado aberto (open), há cada 15 segundos (reset_timeout) o circuito entrará no estado meio-aberto (half-open) para tentar restabelecer a comunicação com o serviço de oferta do parceiro.

Podemos acompanhar (terminal) os eventos do circuito através dos logs da classe LogListener registrada como ouvinte na instancia do circuito.

Antes do circuito invocar a função.
Quando uma invocação de função levanta uma exceção.
Quando o estado do circuito mudou (open).
Quando o estado do circuito mudou (half-open).
Quando o estado do circuito mudou (open).

Caso alteremos o código da resposta do serviço de oferta do parceiro para 200, então o circuito será fechado (close), ou caso a resposta continue com código de erro 500 o circuito continuará aberto.

You might also like...
POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propagation including diffraction
POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propagation including diffraction

POPPY: Physical Optics Propagation in Python POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propaga

Space-invaders - Simple Game created using Python & PyGame, as my Beginner Python Project
Space-invaders - Simple Game created using Python & PyGame, as my Beginner Python Project

Space Invaders This is a simple SPACE INVADER game create using PYGAME whihc hav

Snapchat-filters-app-opencv-python - Here we used opencv and other inbuilt python modules to create filter application like snapchat Yolov5-opencv-cpp-python - Example of using ultralytics YOLO V5 with OpenCV 4.5.4, C++ and Python
Yolov5-opencv-cpp-python - Example of using ultralytics YOLO V5 with OpenCV 4.5.4, C++ and Python

yolov5-opencv-cpp-python Example of performing inference with ultralytics YOLO V

Python-kafka-reset-consumergroup-offset-example - Python Kafka reset consumergroup offset example

Python Kafka reset consumergroup offset example This is a simple example of how

Experimental Python implementation of OpenVINO Inference Engine (very slow, limited functionality). All codes are written in Python. Easy to read and modify.
Experimental Python implementation of OpenVINO Inference Engine (very slow, limited functionality). All codes are written in Python. Easy to read and modify.

PyOpenVINO - An Experimental Python Implementation of OpenVINO Inference Engine (minimum-set) Description The PyOpenVINO is a spin-off product from my

A python-image-classification web application project, written in Python and served through the Flask Microframework
A python-image-classification web application project, written in Python and served through the Flask Microframework

A python-image-classification web application project, written in Python and served through the Flask Microframework. This Project implements the VGG16 covolutional neural network, through Keras and Tensorflow wrappers, to make predictions on uploaded images.

A python-image-classification web application project, written in Python and served through the Flask Microframework. This Project implements the VGG16 covolutional neural network, through Keras and Tensorflow wrappers, to make predictions on uploaded images.
PyArmadillo: an alternative approach to linear algebra in Python

PyArmadillo is a linear algebra library for the Python language, with an emphasis on ease of use.

Comments
  • Bump certifi from 2021.5.30 to 2022.12.7

    Bump certifi from 2021.5.30 to 2022.12.7

    Bumps certifi from 2021.5.30 to 2022.12.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Sourcery Starbot ⭐ refactored jamesgsilva/python-circuit-breaker

    Sourcery Starbot ⭐ refactored jamesgsilva/python-circuit-breaker

    Thanks for starring sourcery-ai/sourcery ✨ 🌟 ✨

    Here's your pull request refactoring your most popular Python repo.

    If you want Sourcery to refactor all your Python repos and incoming pull requests install our bot.

    Review changes via command line

    To manually merge these changes, make sure you're on the main branch, then run:

    git fetch https://github.com/sourcery-ai-bot/python-circuit-breaker main
    git merge --ff-only FETCH_HEAD
    git reset HEAD^
    
    opened by sourcery-ai-bot 0
  • Bump httpx from 0.18.1 to 0.23.0

    Bump httpx from 0.18.1 to 0.23.0

    Bumps httpx from 0.18.1 to 0.23.0.

    Release notes

    Sourced from httpx's releases.

    Version 0.23.0

    0.23.0 (23rd May, 2022)

    Changed

    • Drop support for Python 3.6. (#2097)
    • Use utf-8 as the default character set, instead of falling back to charset-normalizer for auto-detection. To enable automatic character set detection, see the documentation. (#2165)

    Fixed

    • Fix URL.copy_with for some oddly formed URL cases. (#2185)
    • Digest authentication should use case-insensitive comparison for determining which algorithm is being used. (#2204)
    • Fix console markup escaping in command line client. (#1866)
    • When files are used in multipart upload, ensure we always seek to the start of the file. (#2065)
    • Ensure that iter_bytes never yields zero-length chunks. (#2068)
    • Preserve Authorization header for redirects that are to the same origin, but are an http-to-https upgrade. (#2074)
    • When responses have binary output, don't print the output to the console in the command line client. Use output like <16086 bytes of binary data> instead. (#2076)
    • Fix display of --proxies argument in the command line client help. (#2125)
    • Close responses when task cancellations occur during stream reading. (#2156)
    • Fix type error on accessing .request on HTTPError exceptions. (#2158)

    Version 0.22.0

    0.22.0 (26th January, 2022)

    Added

    Fixed

    • Don't perform unreliable close/warning on __del__ with unclosed clients. (#2026)
    • Fix Headers.update(...) to correctly handle repeated headers (#2038)

    Version 0.21.3

    0.21.3 (6th January, 2022)

    Fixed

    • Fix streaming uploads using SyncByteStream or AsyncByteStream. Regression in 0.21.2. (#2016)

    Version 0.21.2

    0.21.2 (5th January, 2022)

    Fixed

    • HTTP/2 support for tunnelled proxy cases. (#2009)
    • Improved the speed of large file uploads. (#1948)

    Version 0.21.1

    ... (truncated)

    Changelog

    Sourced from httpx's changelog.

    0.23.0 (23rd May, 2022)

    Changed

    • Drop support for Python 3.6. (#2097)
    • Use utf-8 as the default character set, instead of falling back to charset-normalizer for auto-detection. To enable automatic character set detection, see the documentation. (#2165)

    Fixed

    • Fix URL.copy_with for some oddly formed URL cases. (#2185)
    • Digest authentication should use case-insensitive comparison for determining which algorithm is being used. (#2204)
    • Fix console markup escaping in command line client. (#1866)
    • When files are used in multipart upload, ensure we always seek to the start of the file. (#2065)
    • Ensure that iter_bytes never yields zero-length chunks. (#2068)
    • Preserve Authorization header for redirects that are to the same origin, but are an http-to-https upgrade. (#2074)
    • When responses have binary output, don't print the output to the console in the command line client. Use output like <16086 bytes of binary data> instead. (#2076)
    • Fix display of --proxies argument in the command line client help. (#2125)
    • Close responses when task cancellations occur during stream reading. (#2156)
    • Fix type error on accessing .request on HTTPError exceptions. (#2158)

    0.22.0 (26th January, 2022)

    Added

    Fixed

    • Don't perform unreliable close/warning on __del__ with unclosed clients. (#2026)
    • Fix Headers.update(...) to correctly handle repeated headers (#2038)

    0.21.3 (6th January, 2022)

    Fixed

    • Fix streaming uploads using SyncByteStream or AsyncByteStream. Regression in 0.21.2. (#2016)

    0.21.2 (5th January, 2022)

    Fixed

    • HTTP/2 support for tunnelled proxy cases. (#2009)
    • Improved the speed of large file uploads. (#1948)

    0.21.1 (16th November, 2021)

    Fixed

    • The response.url property is now correctly annotated as URL, instead of Optional[URL]. (#1940)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Owner
James G Silva
Desenvolvedor de software, ajudo pessoas nos primeiros passos da programação.
James G Silva
tinykernel - A minimal Python kernel so you can run Python in your Python

tinykernel - A minimal Python kernel so you can run Python in your Python

fast.ai 37 Dec 2, 2022
Python-experiments - A Repository which contains python scripts to automate things and make your life easier with python

Python Experiments A Repository which contains python scripts to automate things

Vivek Kumar Singh 11 Sep 25, 2022
Crab is a flexible, fast recommender engine for Python that integrates classic information filtering recommendation algorithms in the world of scientific Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a flexible, fast recommender engine for Python that integrates classic information filtering r

python-recsys 1.2k Dec 21, 2022
Python scripts to detect faces in Python with the BlazeFace Tensorflow Lite models

Python scripts to detect faces using Python with the BlazeFace Tensorflow Lite models. Tested on Windows 10, Tensorflow 2.4.0 (Python 3.8).

Ibai Gorordo 46 Nov 17, 2022
A fast python implementation of Ray Tracing in One Weekend using python and Taichi

ray-tracing-one-weekend-taichi A fast python implementation of Ray Tracing in One Weekend using python and Taichi. Taichi is a simple "Domain specific

null 157 Dec 26, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
This is an open source python repository for various python tests

Welcome to Py-tests This is an open source python repository for various python tests. This is in response to the hacktoberfest2021 challenge. It is a

Yada Martins Tisan 3 Oct 31, 2021
Composable transformations of Python+NumPy programsComposable transformations of Python+NumPy programs

Chex Chex is a library of utilities for helping to write reliable JAX code. This includes utils to help: Instrument your code (e.g. assertions) Debug

DeepMind 506 Jan 8, 2023
Automatic self-diagnosis program (python required)Automatic self-diagnosis program (python required)

auto-self-checker 자동으로 자가진단 해주는 프로그램(python 필요) 중요 이 프로그램이 실행될때에는 절대로 마우스포인터를 움직이거나 키보드를 건드리면 안된다(화면인식, 마우스포인터로 직접 클릭) 사용법 프로그램을 구동할 폴더 내의 cmd창에서 pip

null 1 Dec 30, 2021