ASGI middleware to record and emit timing metrics (to something like statsd)

Related tags

Monitoring timing-asgi

Overview

timing-asgi

This is a timing middleware for ASGI, useful for automatic instrumentation of ASGI endpoints.

This was developed at GRID for use with our backend services which are built using python and the ASGI framework Starlette, and intended to emit metrics to Datadog, a statsd-based cloud monitoring service.

ASGI version

Since 0.2.0 this middleware only supports ASGI3, if you need ASGI2 support please use version 0.1.2.

installation

pip install timing-asgi

usage

Here's an example using the Starlette ASGI framework which prints out the timing metrics..

A more realistic example which emits the timing metrics to Datadog can be found at https://github.com/steinnes/timing-starlette-asgi-example.

import uvicorn

from starlette.applications import Starlette
from starlette.responses import PlainTextResponse
from timing_asgi import TimingMiddleware, TimingClient
from timing_asgi.integrations import StarletteScopeToName


class PrintTimings(TimingClient):
    def timing(self, metric_name, timing, tags):
        print(metric_name, timing, tags)


app = Starlette()


@app.route("/")
def homepage(request):
    return PlainTextResponse("hello world")


app.add_middleware(
    TimingMiddleware,
    client=PrintTimings(),
    metric_namer=StarletteScopeToName(prefix="myapp", starlette_app=app)
)

if __name__ == "__main__":
    uvicorn.run(app)

Running this example and sending some requests:

$ python app.py
INFO: Started server process [35895]
INFO: Waiting for application startup.
2019-03-07 11:38:01 INFO  [timing_asgi.middleware:44] ASGI scope of type lifespan is not supported yet
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: ('127.0.0.1', 58668) - "GET / HTTP/1.1" 200
myapp.__main__.homepage 0.0006690025329589844 ['http_status:200', 'http_method:GET', 'time:wall']
myapp.__main__.homepage 0.0006950000000000012 ['http_status:200', 'http_method:GET', 'time:cpu']
INFO: ('127.0.0.1', 58684) - "GET /asdf HTTP/1.1" 404
myapp.asdf 0.0005478858947753906 ['http_status:404', 'http_method:GET', 'time:wall']
myapp.asdf 0.0005909999999999804 ['http_status:404', 'http_method:GET', 'time:cpu']

Implementation of Common Image Evaluation Metrics by Sayed Nadim (sayednadim.github.io). The repo is built based on full reference image quality metrics such as L1, L2, PSNR, SSIM, LPIPS. and feature-level quality metrics such as FID, IS. It can be used for evaluating image denoising, colorization, inpainting, deraining, dehazing etc. where we have access to ground truth.

Image Quality Evaluation Metrics Implementation of some common full reference image quality metrics. The repo is built based on full reference image q

10 Jan 1, 2023

ASGI middleware for authentication, rate limiting, and building CRUD endpoints.

Piccolo API Utilities for easily exposing Piccolo models as REST endpoints in ASGI apps, such as Starlette and FastAPI. Includes a bunch of useful ASG

81 Dec 9, 2022

Fastapi-auth-middleware - Lightweight auth middleware for FastAPI that just works. Fits most auth workflows with only a few lines of code

FastAPI Auth Middleware We at Code Specialist love FastAPI for its simplicity an

27 Oct 16, 2022

A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Ubelt is a small library of robust, tested, documented, and simple functions that extend the Python standard library. It has a flat API that all behav

638 Dec 13, 2022

A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Ubelt is a small library of robust, tested, documented, and simple functions that extend the Python standard library. It has a flat API that all behav

638 Dec 13, 2022

Metrics-advisor - Analyze reshaped metrics from TiDB cluster Prometheus and give some advice about anomalies and correlation.

metrics-advisor Analyze reshaped metrics from TiDB cluster Prometheus and give some advice about anomalies and correlation. Team freedeaths mashenjun

3 Jan 7, 2022

TorchMetrics is a collection of 25+ PyTorch metrics implementations and an easy-to-use API to create custom metrics.

Machine learning metrics for distributed, scalable PyTorch applications.

1.2k Jan 6, 2023

FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python

☑️ FAIR Enough metrics for research FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python, conforming to the specifications

3 Jul 6, 2022

Exactly what it sounds like, which is something rad

EyeWitnessTheFitness External recon got ya down? That scan prevention system preventing you from enumerating web pages? Well look no further, I have t

18 Dec 31, 2022

Something like Asteroids but not really, done in CircuitPython

CircuitPython Staroids Something like Asteroids, done in CircuitPython. Works with FunHouse, MacroPad, Pybadge, EdgeBadge, CLUE, and Pygamer. circuitp

14 May 31, 2022

Hypothesis strategies for generating Python programs, something like CSmith

hypothesmith Hypothesis strategies for generating Python programs, something like CSmith. This is definitely pre-alpha, but if you want to play with i

73 Dec 14, 2022

Stock-history-display - something like a easy yearly review for your stock performance

Stock History Display Available on Heroku: https://stock-history-display.herokua

1 Jan 7, 2022

A Python application for tracking, reporting on timing and complexity in Python code

A command-line application for tracking, reporting on complexity of Python tests and applications. wily [a]: quick to think of things, having a very g

1k Dec 29, 2022

A task scheduler with task scheduling, timing and task completion time tracking functions

A task scheduler with task scheduling, timing and task completion time tracking functions. Could be helpful for time management in daily life.

0 Jan 15, 2022

CPSPEC is an astrophysical data reduction software for timing

CPSPEC manual Introduction CPSPEC is an astrophysical data reduction software for timing. Various timing properties, such as power spectra and cross s

1 Oct 20, 2021

ASGI specification and utilities

asgiref ASGI is a standard for Python asynchronous web apps and servers to communicate with each other, and positioned as an asynchronous successor to

1.1k Dec 29, 2022

Drop-in MessagePack support for ASGI applications and frameworks

msgpack-asgi msgpack-asgi allows you to add automatic MessagePack content negotiation to ASGI applications (Starlette, FastAPI, Quart, etc.), with a s

128 Jan 2, 2023

Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

300 Dec 29, 2022

Fast⚡, simple and light💡weight ASGI micro🔬 web🌏-framework for Python🐍.

NanoASGI Asynchronous Python Web Framework NanoASGI is a fast ⚡ , simple and light 💡 weight ASGI micro 🔬 web 🌏 -framework for Python 🐍 . It is dis

8 Jun 16, 2022

Comments

Windows Support Added

Added support for windows - resource module is not imported, warning raised and cpu time hard coded to 0.

I doubt windows would ever be used in production but for local debugging, allowing Windows to import the module makes development less painful.

#18

opened by cmdev-ls 5
Rename project

I decided to make this a little more generic than I originally intended, and the name statsd-asgi was not descriptive enough anymore. This is a general timing middleware, and the timings could be emitted as something other than statsd down the line.

opened by steinnes 0
ModuleNotFoundError: No module named 'resource' on Windows

get_cpu_time() uses the Unix only resource package. Instead of raising an exception, I'd expect a warning message and a different method of calculating cpu time, or even set cpu_time to 0.

opened by mvanderlee 1
Metrics include time spent on background tasks in Starlette

For some routes, we make use of the background tasks feature in Starlette. The timing metrics we have gathered indicate that time spent executing these background tasks is being included. While that's not necessarily bad, it would be preferable if we could have a measure of response time separate from overall time spent on a request.
bug

opened by eirikur-grid 3

ASGI middleware to record and emit timing metrics (to something like statsd)

Related tags

Overview

timing-asgi

ASGI version

installation

usage

You might also like...

ASGI middleware for authentication, rate limiting, and building CRUD endpoints.

Fastapi-auth-middleware - Lightweight auth middleware for FastAPI that just works. Fits most auth workflows with only a few lines of code

A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

A Python utility belt containing simple tools, a stdlib like feel, and extra batteries. Hashing, Caching, Timing, Progress, and more made easy!

Metrics-advisor - Analyze reshaped metrics from TiDB cluster Prometheus and give some advice about anomalies and correlation.

TorchMetrics is a collection of 25+ PyTorch metrics implementations and an easy-to-use API to create custom metrics.

FAIR Enough Metrics is an API for various FAIR Metrics Tests, written in python

Exactly what it sounds like, which is something rad

Something like Asteroids but not really, done in CircuitPython

Hypothesis strategies for generating Python programs, something like CSmith

Stock-history-display - something like a easy yearly review for your stock performance

A Python application for tracking, reporting on timing and complexity in Python code

A task scheduler with task scheduling, timing and task completion time tracking functions

CPSPEC is an astrophysical data reduction software for timing

ASGI specification and utilities

Drop-in MessagePack support for ASGI applications and frameworks

Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Fast⚡, simple and light💡weight ASGI micro🔬 web🌏-framework for Python🐍.

Comments

Windows Support Added

Rename project

ModuleNotFoundError: No module named 'resource' on Windows

Metrics include time spent on background tasks in Starlette

Owner

Steinn Eldjárn Sigurðarson

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

Real-time metrics for nginx server

Exports osu! user stats to prometheus metrics for a specified set of users

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

Cross-platform lib for process and system monitoring in Python

Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems.

Development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application.

Scalene: a high-performance, high-precision CPU and memory profiler for Python

asgi-server-timing-middleware

Diamond is a python daemon that collects system metrics and publishes them to Graphite (and others). It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.