Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

Overview

Build Status Python PyPI version codecov Docs Downloads Language grade: Python

starlette context

Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

Resources:

Installation

$ pip install starlette-context

Requirements

Python 3.7+

Dependencies

  • starlette

All other dependencies from requirements-dev.txt are only needed to run tests or examples. Test/dev env is dockerized if you want to try them yourself.

Example

import uvicorn

from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import JSONResponse

from starlette_context import context, plugins
from starlette_context.middleware import RawContextMiddleware

middleware = [
    Middleware(
        RawContextMiddleware,
        plugins=(
            plugins.RequestIdPlugin(),
            plugins.CorrelationIdPlugin()
        )
    )
]

app = Starlette(middleware=middleware)


@app.route("/")
async def index(request: Request):
    return JSONResponse(context.data)


uvicorn.run(app, host="0.0.0.0")

In this example the response contains a json with

{
  "X-Correlation-ID":"5ca2f0b43115461bad07ccae5976a990",
  "X-Request-ID":"21f8d52208ec44948d152dc49a713fdd"
}

Context can be updated and accessed at anytime if it's created in the middleware.

Contribution

See the guide on read the docs.

Comments
  • Allow custom validation response

    Allow custom validation response

    This PR fixes 2 issues:

    • A user sending invalid data in a UUID-based plugin should not result in a server error, but a client error with a 400 status code.
    • Middlewares in Starlette should not raise Exceptions, they should abort the request cycle and send a response instead.

    The exact error is customizable from user code, but provides a default 400 response with empty body. Implemented on both ContextMiddleware and RawContextMiddleware.

    enhancement 
    opened by hhamana 11
  • Testing code that relies on context vars without a full test client / app

    Testing code that relies on context vars without a full test client / app

    Sometimes it can be very useful to write tests for functions that rely on some data in the request context, but without using a full Starlette test client, as tests that are based on test clients are much less "lean" and do a lot of things beyond just running the code unit under test.

    I think it would be very useful to document / add some helpers that enable a context manager that mocks a request / response cycle / middleware / etc. so it can be used in testing.

    For now, I have done this in my tests (I'm using pytest fixtures, this is a very simplified example):

    code under test

    from starlette_context import context
    
    def get_session_id():
        """Get the current session ID"""
        return context['session_id']
    

    in conftest.py:

    import pytest
    from starlette_context import _request_scope_context_storage, context
    
    
    @pytest.fixture()
    def request_context():
        token = _request_scope_context_storage.set({})
        try:
            yield context
        finally:
            _request_scope_context_storage.reset(token)
    

    in tests:

    from myapp.session_utils import get_session_id 
    
    
    def test_some_func_that_relies_on_context(request_context):
        request_context['session_id'] = 'foobar'
        assert get_session_id() == 'foobar'
    

    Obviously, this is a bit hackish and it would be nice if there would be an official / documented way of doing this without accessing module internals.

    question 
    opened by shevron 8
  • Are there plans to add type hints or library stubs for MyPy?

    Are there plans to add type hints or library stubs for MyPy?

    I'm using your library right now in a FastAPI app, and since everything's heavily typed in FastAPI (and Pydantic), I have MyPy enabled and it's using quite a strict configuration. Right now, it's complaining about the imports from starlette_context:

    Code:

    from starlette_context import context
    from starlette_context.plugins import Plugin, RequestIdPlugin
    

    Error:

    Skipping analyzing 'starlette_context': found module but no type hints or library stubs  [import]
    Skipping analyzing 'starlette_context.plugins': found module but no type hints or library stubs  [import]
    See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
    

    I can always just suppress the error with # type: ignore but I was wondering if you were planning on adding type hints or stub packages/files, as recommended by MyPy here: https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-type-hints-for-third-party-library. I think that would be a better approach.

    In case there's no plan for this yet, what would you say to a PR to add stub files? Basically, I think just adding the appropriate .pyi files next to each .py file would satisfy MyPy.

    invalid wontfix 
    opened by ginomempin 8
  • uninitialized context access raises LookupError

    uninitialized context access raises LookupError

    Here is a testcase:

    >>> from starlette_context import context
    >>> context
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/dmig/.pyenv/versions/3.8.2/lib/python3.8/collections/__init__.py", line 1014, in __repr__
        def __repr__(self): return repr(self.data)
      File "/home/dmig/.pyenv/versions/marty-services-3.8.2/lib/python3.8/site-packages/starlette_context/ctx.py", line 26, in data
        return _request_scope_context_storage.get()
    LookupError: <ContextVar name='starlette_context' at 0x7f3b3be81c20>
    
    enhancement invalid question 
    opened by dmig-alarstudios 8
  • ContextDoesNotExistError on FastAPI Custom Exception Handler

    ContextDoesNotExistError on FastAPI Custom Exception Handler

    Hi, i'm using Depend FastAPI mechaninsm to access Context on request-response cycle:

    async def my_context_dependency(
        x_plt_session_id: str = Header(None),
        x_plt_correlation_id: str = Header(None),
        x_plt_user_id: str = Header(None),
        x_plt_event_id: str = Header(None),
        x_plt_solution_user: str = Header(None),
    ) -> Any:
        # When used a Depends(), this fucntion get the `X-Client_ID` header,
        # which will be documented as a required header by FastAPI.
        # use `x_client_id: str = Header(None)` for an optional header.
    
        data = {
            "session-id": x_plt_session_id,
            "correlation-id": x_plt_correlation_id,
            "user-id": x_plt_user_id,
            "event-id": x_plt_event_id,
            "solution-user": x_plt_solution_user,
        }
        with request_cycle_context(data):
            # yield allows it to pass along to the rest of the request
            yield
    
    app = FastAPI(
            dependencies=[Depends(my_context_dependency)],
            title="Virtual Entity API",
            description="This is a very fancy project, with auto docs for the API and everything",
            version="0.0.1",
            openapi_url="/openapi.json"
        )
    

    I'm also using custom exception handler in FastAPI:

    from starlette_context import context
    
    @app.exception_handler(AWSException)
    async def unicorn_exception_handler(
        request: Request, exc: AWSException
    ) -> JSONResponse:
    
       user_id = context.get("user-id", default=None)
        
        print(user_id )
    
        return JSONResponse(
            status_code=400,
            content={"message": exc.message},
        )
    

    But when i try to access context inside the custom exception handler i receive :

    starlette_context.errors.ContextDoesNotExistError: You didn't use the required middleware or you're trying to access context object outside of the request-response cycle.

    Any hints ?

    enhancement invalid 
    opened by mancioshell 7
  • `starlette_context` context manager

    `starlette_context` context manager

    This creates and exposes a context manager that instantiates and resets the Token.

    This isn't much actual code, it may look like it only slightly improves the code reuse of the token initialization/reset code, but exposing it while abstracting the internal details allows 2 more important use cases:

    • within unit tests, creating a mock environment to test a portion using the context out of a request-response cycle (helping for #46).
    • leveraging FastAPI framework Depends() to allow required/optional headers to be visibly documented, plus use the full scope of FastAPI features (such as easy access to the request's body within a Depends system, (as #50 shown might be a use case).

    This also introduces the first explicit support of FastAPI, which, while limited, may attract more attention to the library.

    documentation enhancement 
    opened by hhamana 6
  • rewrite middleware to pure ASGI

    rewrite middleware to pure ASGI

    Since there are issues in Starlette caused by BaseHTTPMiddleware class, this package becomes a source of these issues in any project using it:

    • https://github.com/encode/starlette/issues/919
    • https://github.com/encode/starlette/issues/1012

    The simple solution would be to rewrite the middleware to pure ASGI.

    enhancement 
    opened by dmig-alarstudios 5
  • PluginUUIDBase's force_new_uuid option seems broken

    PluginUUIDBase's force_new_uuid option seems broken

    When using PluginUUIDBase's force_new_uuid option and setting it to True I'm always getting the same uuid. Which is the opposite of what I expect.

    It looks like when self.value is None a new uuid is generated, and this works fine when for the first request. But subsequent request seem to be using the same id.

    I suspect the code that needs to be fixed is missing a self.value = None before trying anything else here: https://github.com/tomwojcik/starlette-context/blob/2f80262bbf4c00fb501c22ac04a5c1c408187fe6/starlette_context/plugins/plugin_uuid.py#L34

    bug 
    opened by wapiflapi 5
  • Support for CorrelationIdPlugin and CorrelationIdPlugin still executing when application encounters an exception

    Support for CorrelationIdPlugin and CorrelationIdPlugin still executing when application encounters an exception

    Issue

    When a route raises an exception and so returns a 500 Internal Server Error, x-request-id and x-correlation-id are not set.

    The culprit seems to be https://github.com/tomwojcik/starlette-context/blob/79aa8ffe5b50db2263e2073fc5d252bf442f150c/starlette_context/middleware.py#L46-L52

    This is similar to this comment - https://github.com/tiangolo/fastapi/issues/397#issuecomment-543136587

    Expectation I'd expect those headers to be set for all responses. One benefit of correlation ids is when there are errors, we can use the ids to track down the issue.

    Steps to reproduce Sample. The "/" endpoint returns the x-request-id and x-correlation-id headers. The "/error" does not.

    from starlette.applications import Starlette
    from starlette.middleware import Middleware
    from starlette.requests import Request
    from starlette.responses import JSONResponse
    
    import uvicorn
    from starlette_context import context, plugins
    from starlette_context.middleware import ContextMiddleware
    
    middleware = [
        Middleware(
            ContextMiddleware,
            plugins=(plugins.RequestIdPlugin(), plugins.CorrelationIdPlugin()),
        )
    ]
    
    app = Starlette(debug=True, middleware=middleware)
    
    
    @app.route("/")
    async def index(request: Request):
        return JSONResponse(context.data)
    
    @app.route("/error")
    async def index(request: Request):
        raise RuntimeError()
        return JSONResponse(context.data)
    
    uvicorn.run(app, host="0.0.0.0")
    
    documentation question 
    opened by derekbekoe 5
  • Can't write new plugin - plugin is not iterable

    Can't write new plugin - plugin is not iterable

    Hi!

    I am quite new to Starlette and FastAPI, and I might have holes in my knowledge, however I was not able to write a new plugin.

    from fastapi import FastAPI
    from fastapi.middleware import Middleware
    
    from starlette_context.header_keys import HeaderKeys
    from starlette_context.plugins import Plugin
    
    class LangPlugin(Plugin):
        print('****')
    
    middleware = [
        Middleware(
            RawContextMiddleware,
            plugins(
                LangPlugin()
            )
        )
    ]
    
    app = FastAPI(middleware=middleware)
    

    And it says:

      File ".../starlette_context/middleware/raw_middleware.py", line 17, in __init__
        if not all([isinstance(plugin, Plugin) for plugin in self.plugins]):
    TypeError: 'LangPlugin' object is not iterable
    

    How is my plugin not instance of Plugin, I wonder. Or I guess this is the problem.

    To put you into context, I would like to get from here an optional string param, (/path-to-api-endpoint/?lang=en) to access the request language from everywhere. Or maybe you have a more straightforward solution to my problem, what I did not recognize?

    invalid question 
    opened by dddenes 4
  • Add request-id to logs without using LoggerAdapter

    Add request-id to logs without using LoggerAdapter

    Thank you for this middleware!

    I want to use it for adding request IDs in the logs of third-party libraries that I do not have control over. Since most libraries might use a logger instead of LoggerAdapter, I am wondering if there is a way to log request IDs without the use LoggerAdapter(like in the example given in this repo).

    documentation 
    opened by dev-99 4
  • WIP: Test where asgi app is spammed with requests and the context (response headers) is always correct

    WIP: Test where asgi app is spammed with requests and the context (response headers) is always correct

    @hhamana I tested your PR. It works fine. I'm worried about regressions where context is not cleared though.

    I'd like to add a test where the client is spamming asgi app with requests. In the meantime, half of them fails. If we can check if response headers are correct after that, it should prove there are no regressions in real world apps with huge traffic.

    The only thing left is to figure out why await asyncio.sleep is not working.

    help wanted 
    opened by tomwojcik 0
  • Properly use pyproject.toml

    Properly use pyproject.toml

    The pyproject.toml we use is currently only for black config. While that's fine and all, pyproject.toml is originally intended to define the build system, and replace setup.cfg, which itself is already supposed to be a safer alternative to the setup.py As build system, using the status quo default setuptools is fine for us. But we have to be explicit about it. Latest updates to pip issue a DeprecatedWarning to systems still using setup.py or setup.cfg to define project metadata.

    recommended reading:

    • https://snarky.ca/what-the-heck-is-pyproject-toml/
    • https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
    dependencies github_actions 
    opened by hhamana 2
  • drop error_response, add error detail and support for serialization

    drop error_response, add error detail and support for serialization

    Closes https://github.com/tomwojcik/starlette-context/issues/58

    bug enhancement 
    opened by tomwojcik 6
  • Improved logging

    Improved logging

    Not sure if I was doing something wrong but it seems like the library could provide more useful logging. For instance I was passing non valid UUID's as the X-Request-ID and in this case all the logging I saw on my server was:

    127.0.0.1:51990 - "POST /graphql/ HTTP/1.1" 400
    

    Some descriptive log message would help.

    bug enhancement 
    opened by KBoehme 4
  • Consider removing `RawContextMiddleware` and keeping only `ContextMiddleware` after resolving memory usage issue

    Consider removing `RawContextMiddleware` and keeping only `ContextMiddleware` after resolving memory usage issue

    I like ContextMiddleware because it's very simple to understand and expand, even for minds that are not familiar with async Python.

    Some time ago it was advised https://github.com/tomwojcik/starlette-context/issues/18 to add RawContextMiddleware and Starlette maintainers were discouraging the use of the built-in middleware. I was surprised to see 3 ❤️ reactions under this issue so I think a few people had problems with it.

    Fast-forward almost a year, the issue https://github.com/encode/starlette/issues/1012#issuecomment-866622798 with memory usage has been closed.

    If there's no point in keeping both, I'd be happy to remove the more complicated one but I will wait for some confirmations from the community. If there's a case when it makes sense to use RawContextMiddleware, then I'd keep both.

    What's your opinion about that? @hhamana @dmig-alarstudios

    question 
    opened by tomwojcik 8
Releases(v0.3.5)
  • v0.3.5(Nov 26, 2022)

  • v0.3.4(Jun 22, 2022)

    • add request_cycle_context. It’s a context manager that allows for easier testing and cleaner code (Thanks @hhamana) https://github.com/tomwojcik/starlette-context/issues/46
    • fix for accessing context during logging, outside of the request-response cycle. Technically it should raise an exception, but it makes sense to include the context by default (in logs) and if it’s not available, some logs are better than no logs. Now it will show context data if context is available, with a fallback to an empty dict (instead of raising an exc) https://github.com/tomwojcik/starlette-context/issues/65
    • add ContextMiddleware deprecation warning
    • **context context unpacking seems to be working now
    Source code(tar.gz)
    Source code(zip)
  • v0.3.3(Jun 28, 2021)

    • add support for custom error responses if error occurred in plugin / middleware -> fix for 500 (Thanks @hhamana)
    • better (custom) exceptions with a base StarletteContextError (Thanks @hhamana)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Apr 22, 2021)

    • ContextDoesNotExistError is raised when context object can't be accessed. Previously it was RuntimeError. For backwards compatibility, it inherits from RuntimeError so it shouldn't result in any regressions.
    • Added py.typed file so your mypy should never complain
    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Oct 17, 2020)

  • v0.3.0(Oct 10, 2020)

    • add RawContextMiddleware for Streaming and File responses
    • add flake8, isort, mypy
    • small refactor of the base plugin, moved directories and removed one redundant method (potentially breaking changes)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Jul 27, 2020)

    • add docs on read the docs
    • fix bug with force_new_uuid=True returning the same uuid constantly
    • due to ^ a lot of tests had to be refactored as well
    Source code(tar.gz)
    Source code(zip)
  • 0.2.2(Apr 26, 2020)

    • for correlation id and request id plugins, add support for enforcing the generation of a new value
    • for ^ plugins add support for validating uuid. It's a default behavior so will break things for people who don't use uuid4 there. If you don't want this validation, you need to pass validate=False to the plugin
    • thanks to @VukW you can now check if context is available
    Source code(tar.gz)
    Source code(zip)
  • 0.2.1(Apr 18, 2020)

    • dropped with_plugins from the middleware as Starlette has it's own way of doing this
    • due to ^ this change some tests are simplified
    • if context is not available no LookupError will be raised, instead there will be RuntimeError, because this error might mean one of two things: user either didn't use ContextMiddleware or is trying to access context object outside of request-response cycle
    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Feb 21, 2020)

    • changed parent of context object. More or less the API is the same but due to this change the implementation itself is way more simple and now it's possible to use .items() or keys() like in a normal dict, out of the box. Still, unpacking **kwargs is not supported and I don't think it ever will be. I tried to inherit from the builtin dict but nothing good came out of this. Now you access context as dict using context.data, not context.dict()
    • there was an issue related to not having awaitable plugins. Now both middleware and plugins are fully async compatible. It's a breaking change as it forces to use await, hence new minor version
    Source code(tar.gz)
    Source code(zip)
  • 0.1.6(Jan 2, 2020)

  • 0.1.5(Jan 1, 2020)

  • 0.1.4(Dec 31, 2019)

  • 0.1.3(Dec 30, 2019)

  • 0.1.2(Dec 30, 2019)

  • 0.1.1(Dec 28, 2019)

  • 0.1(Dec 27, 2019)

Owner
Tomasz Wójcik
There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.
Tomasz Wójcik
Prometheus integration for Starlette.

Starlette Prometheus Introduction Prometheus integration for Starlette. Requirements Python 3.6+ Starlette 0.9+ Installation $ pip install starlette-p

José Antonio Perdiguero 229 Dec 21, 2022
Automatically monitor the evolving performance of Flask/Python web services.

Flask Monitoring Dashboard A dashboard for automatic monitoring of Flask web-services. Key Features • How to use • Live Demo • Feedback • Documentatio

null 663 Dec 29, 2022
🚴 Call stack profiler for Python. Shows you why your code is slow!

pyinstrument Pyinstrument is a Python profiler. A profiler is a tool to help you 'optimize' your code - make it faster. It sounds obvious, but to get

Joe Rickerby 5k Jan 1, 2023
GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

GoAccess What is it? GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through y

Gerardo O. 15.6k Jan 2, 2023
Cross-platform lib for process and system monitoring in Python

Home Install Documentation Download Forum Blog Funding What's new Summary psutil (process and system utilities) is a cross-platform library for retrie

Giampaolo Rodola 9k Jan 2, 2023
Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems.

Glances - An eye on your system Summary Glances is a cross-platform monitoring tool which aims to present a large amount of monitoring information thr

Nicolas Hennion 22k Jan 4, 2023
Development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application.

README for pympler Before installing Pympler, try it with your Python version: python setup.py try If any errors are reported, check whether your Pyt

null 996 Jan 1, 2023
Scalene: a high-performance, high-precision CPU and memory profiler for Python

scalene: a high-performance CPU and memory profiler for Python by Emery Berger 中文版本 (Chinese version) About Scalene % pip install -U scalene Scalen

Emery Berger 138 Dec 30, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 300 Dec 26, 2022
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 110 Feb 16, 2021
Markup is an online annotation tool that can be used to transform unstructured documents into structured formats for NLP and ML tasks, such as named-entity recognition. Markup learns as you annotate in order to predict and suggest complex annotations. Markup also provides integrated access to existing and custom ontologies, enabling the prediction and suggestion of ontology mappings based on the text you're annotating.

Markup is an online annotation tool that can be used to transform unstructured documents into structured formats for NLP and ML tasks, such as named-entity recognition. Markup learns as you annotate in order to predict and suggest complex annotations. Markup also provides integrated access to existing and custom ontologies, enabling the prediction and suggestion of ontology mappings based on the text you're annotating.

Samuel Dobbie 146 Dec 18, 2022
Fully Automated YouTube Channel ▶️with Added Extra Features.

Fully Automated Youtube Channel ▒█▀▀█ █▀▀█ ▀▀█▀▀ ▀▀█▀▀ █░░█ █▀▀▄ █▀▀ █▀▀█ ▒█▀▀▄ █░░█ ░░█░░ ░▒█░░ █░░█ █▀▀▄ █▀▀ █▄▄▀ ▒█▄▄█ ▀▀▀▀ ░░▀░░ ░▒█░░ ░▀▀▀ ▀▀▀░

sam-sepiol 249 Jan 2, 2023
Dead simple CSRF security middleware for Starlette ⭐ and Fast API ⚡

csrf-starlette-fastapi Dead simple CSRF security middleware for Starlette ⭐ and Fast API ⚡ Will work with either a <input type="hidden"> field or ajax

Nathaniel Sabanski 9 Nov 20, 2022
Code Specialist 27 Oct 16, 2022
Starlette middleware for Prerender

Prerender Python Starlette Starlette middleware for Prerender Documentation: https://BeeMyDesk.github.io/prerender-python-starlette/ Source Code: http

BeeMyDesk 14 May 2, 2021
Starlette middleware for Prerender

Prerender Python Starlette Starlette middleware for Prerender Documentation: https://BeeMyDesk.github.io/prerender-python-starlette/ Source Code: http

BeeMyDesk 13 Jul 27, 2020
Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

Intellia ICT 5 Oct 30, 2022