ASGI middleware for authentication, rate limiting, and building CRUD endpoints.

Overview

Build Status

Documentation Status

Coverage Status

Piccolo API

Utilities for easily exposing Piccolo models as REST endpoints in ASGI apps, such as Starlette and FastAPI.

Includes a bunch of useful ASGI middleware:

  • Session Auth
  • Token Auth
  • Rate Limiting
  • CSRF
  • Content Security Policy (CSP)
  • And more

You can read the docs here.

Issues
  • added ability to request a subset of columns

    added ability to request a subset of columns

    Related to #97

    opened by sinisaos 23
  • Add a complete session auth example to the docs

    Add a complete session auth example to the docs

    It would be useful to show an app which has all of the session auth components working together (session_login, session_logout and SessionsAuthBackend).

    Here's an example:

    import datetime
    from fastapi import FastAPI
    from piccolo_api.csrf.middleware import CSRFMiddleware
    from piccolo_api.openapi.endpoints import swagger_ui
    from piccolo_api.session_auth.endpoints import session_login, session_logout
    from piccolo_api.session_auth.middleware import SessionsAuthBackend
    from starlette.middleware import Middleware
    from starlette.middleware.authentication import AuthenticationMiddleware
    from starlette.routing import Route
    
    app = FastAPI()
    
    app.mount(
        "/login/",
        session_login(),
    )
    
    private_app = FastAPI(
        routes=[
            Route("/logout/", session_logout()),
        ],
        middleware=[
            Middleware(
                AuthenticationMiddleware,
                backend=SessionsAuthBackend(
                    increase_expiry=datetime.timedelta(minutes=30)
                ),
            ),
            Middleware(CSRFMiddleware, allow_form_param=True),
        ],
        docs_url=None,
        redoc_url=None,
    )
    
    # The Swagger docs which come with FastAPI don't support CSRF middleware, so we mount
    # a custom one which Piccolo provides (accessible at /private/docs):
    private_app.mount("/docs/", swagger_ui(schema_url="/private/openapi.json"))
    
    @private_app.get('/my-secret-endpoint/')
    def my_endpoint():
        # This is just a normal FastAPI endpoint, and is protected by Session Auth
        pass
    
    app.mount("/private/", private_app)
    
    if __name__ == "__main__":
        import uvicorn
        uvicorn.run(app)
    
    documentation 
    opened by dantownsend 14
  • Cursor pagination improvements

    Cursor pagination improvements

    Added the ability to move in different directions (forward and backward) from next_cursor.

    Example:
    http://localhost:8000/posts/?__page_size=3&__order=id&__cursor=NA== (ASC forward)
    http://localhost:8000/posts/?__page_size=3&__order=id&__cursor=NA==&__previous=yes (ASC backward)
    http://localhost:8000/posts/?__page_size=3&__order=-id&__cursor=OQ== (DESC forward)
    http://localhost:8000/posts/?__page_size=3&__order=-id&__cursor=OQ==&__previous=yes (DESC backward)
    

    This is quite tricky with a lot of edge cases. I don't know exactly how to use cursor pagination in piccolo_admin. LimitOffset pagination works good (up to 200,000 rows), and is quite convinient. We may use cursor pagination above that number (maybe millions of rows, but I’m have never encountered such large data sets), but then we lose the ability to sort by all columns except by id, because the cursor must be unique. You probably know much better than I how to implement that. Cheers.

    opened by sinisaos 9
  • Troubles with Session Auth

    Troubles with Session Auth

    I am struggling a bit with Piccolo's authentication systems. I have a FastAPI app and am wrapping it with Starlette's AuthenticationMiddleware, as hinted in the docs, with the joint SessionAuth and SecretTokenAuth providers. The secret token seems to be working alright; my API client won't get results without the correct header-cum-token. However whatever I do on the browser gives me "Auth failed for all backends". I can't get to the login endpoint, though this appears properly configured according to the docs. I tried 'allow unauthenticated' to see if this would loosen up the permissions, but even the FastAPI docs give me this error. Is there any robust example app with SessionAuth to see how everything should be organized?

    opened by wmshort 9
  • Improve limit and offset in PiccoloCRUD

    Improve limit and offset in PiccoloCRUD

    Currently, the __page and __page_size query params aren't documented.

    Also, PiccoloCRUD should have a max_page_size argument, to limit abuse of an endpoint.

    If the max_page_size is exceeded, an error should be returned. A 403 feels most appropriate, with a body such as The page size limit has been exceeded.

    enhancement 
    opened by dantownsend 9
  • Add option to exclude columns when creating pydantic model

    Add option to exclude columns when creating pydantic model

    Simple change which adds exclude_columns argument to the pydantic creator. The specified exclude_columns are then skipped in the main for loop.

    opened by kucera-lukas 8
  • Be able to request a subset of columns from PiccoloCRUD's GET endpoint

    Be able to request a subset of columns from PiccoloCRUD's GET endpoint

    For example:

    /api/tables/movie?__fields=id,name
    

    Would return:

    {
        "rows": [
            {
                "id": 17,
                "name": "The Hobbit: The Battle of the Five Armies",
            },
         ]
    }
    

    This is generally useful, but would be great for Piccolo Admin too, as when a user uses a TableConfig to specify that the list page should only show a subset of columns, we aren't over fetching data.

    enhancement 
    opened by dantownsend 8
  • session_logout to use out of the box

    session_logout to use out of the box

    Based on #82

    opened by sinisaos 7
  • Missing new endpoint

    Missing new endpoint

    After adding FastAPI as backend /table/new endpoint return 404 error. Adding missing new endpoint for creating rows in Piccolo Admin.

    opened by sinisaos 6
  • SessionsAuthBackend.authenticate honor allow_unauthenticated if not token

    SessionsAuthBackend.authenticate honor allow_unauthenticated if not token

    Hello, sorry if this is a naive assertion, but shouldn't SessionsAuthBackend.authenticate return (AuthCredentials(scopes=[]), UnauthenticatedUser()) if not token but self.allow_unauthenticated == True?

    bug 
    opened by Bakz 6
  • added new null operator

    added new null operator

    Related to #104

    opened by sinisaos 0
  • added bulk delete option

    added bulk delete option

    Related to #11

    opened by sinisaos 1
  • Add a new operator type of 'null'

    Add a new operator type of 'null'

    There's a limitation at the moment with filtering via PiccoloCRUD - you can't specify 'null' for some column types.

    Take this as an example:

    Screenshot 2021-11-11 at 08 25 13

    The user can't type in 'null'. In Piccolo Admin, a solution is to add 'null' as an option in the operator dropdown (similar to 'Equals' etc).

    When PiccoloCRUD receives an operator value of 'null' it should ignore the value it receives, and treat it as null instead.

    enhancement 
    opened by dantownsend 3
  • another approach to cursor pagination

    another approach to cursor pagination

    This is another approach to cursor pagination by moving the cursor logic from the crud endpoint to a separate file and created the CursorPagination class

    opened by sinisaos 15
  • Add hooks to `session_login`

    Add hooks to `session_login`

    When using the session_login endpoint, it would be nice to have hooks (basically callback functions) which are called at different parts of the verification process.

    There are many situations where this would be valuable. Here are some examples:

    1. You only want to let the user login if they have verified their email address.
    2. After the user logs in, you want to record this in an external system.
    3. Before logging a user in, you want to check if they are on a black list (for example, based on email address or IP address).

    There will have to be a minimum of three hook types:

    1. Triggered before a login attempt is made.
    2. Triggered after the login is successful.
    3. Triggered when the login fails.

    The session_login function will have a new hooks argument:

    def session_login(
        ...,
        hooks: t.List[SessionLoginHook] = []
    ):
        ...
    

    The hook will accept a request argument, and additional data (such as email, BaserUser instance etc).

    If the hook raises an Exception, then the login is stopped, and an error is shown to the user.

    It would be great if this functionality was available for all of the auth types that we provide (token auth etc), but session auth is the most important.

    enhancement good first issue 
    opened by dantownsend 0
  • Create a decorator for wrapping views / endpoints in a database transaction

    Create a decorator for wrapping views / endpoints in a database transaction

    Based on this discussion:

    https://github.com/piccolo-orm/piccolo/discussions/185

    In some frameworks, such as Django, views are wrapped with transaction handling logic. If an exception is raised, then the transaction is rolled back.

    To use transactions is more explicit currently in Piccolo - a context manager has to be used within the view itself.

    @app.get("/")
    async def my_endpoint():
       try:
            async with MyTable._meta.db.transaction():
                # a bunch of queries
       except Exception:
            return JSONResponse({"message": "something went wrong"})
    
        return JSONResponse({"message": "success"})
    

    It would be nice to have a decorator which wraps a view with this logic:

    @app.get("/")
    @piccolo_transaction
    async def my_endpoint():
        ...
    
    enhancement 
    opened by dantownsend 0
  • Make Piccolo API work gracefully if asyncpg isn't installed

    Make Piccolo API work gracefully if asyncpg isn't installed

    Related to:

    https://github.com/piccolo-orm/piccolo_api/pull/72

    If asyncpg isn't installed, Piccolo API won't work:

    https://github.com/piccolo-orm/piccolo_api/blob/4e4d55dc39b978f3d1a8f65b8e2974997b03b528/piccolo_api/crud/serializers.py#L9

    As asyncpg is now an optional requirement in Piccolo, we need to work out a way of making Piccolo API handle it gracefully if asyncpg isn't installed.

    enhancement 
    opened by dantownsend 0
  • Correctly add default values to created pydantic models

    Correctly add default values to created pydantic models

    Right now default values are not being added (it's either ellipsis or None if we pass in all_optional as True). This is an issue because if people depend on the created pydantic schemas in their api endpoints the default values will not be parsed and will not be shown in openapi spec (eg. pydantic and fastapi schema generation).

    opened by kucera-lukas 7
  • Implement cursor based pagination in PiccoloCRUD

    Implement cursor based pagination in PiccoloCRUD

    Currently, only offset based pagination is supported in PiccoloCRUD (using the __page and __page_size query params). There are issues with this, as outlined in this article.

    Modify PiccoloCRUD so it accepts a __cursor GET parameter. The value will be the ID of a row for now, but encoded as string. Encoding it as a string gives more flexibility (for example, we may want to return a UUID instead of an integer in the future).

    The response should then be something like:

    {
        "rows": [...],
        "next_cursor: "1234"
    }
    

    If both the __page and __cursor GET params are passed to an endpoint, an error should be returned.

    enhancement 
    opened by dantownsend 7
  • Be able to filter bulk DELETE and GET queries by a list of row IDs

    Be able to filter bulk DELETE and GET queries by a list of row IDs

    Currently, PiccoloCRUD (and hence FastAPIWrapper) allow you to filter by the row ID, but you can't filter by a list of IDs.

    For example:

    # Currently supported:
    DELETE /api/tables/movies?id=1
    
    # Proposed:
    DELETE /api/tables/movies?id=1,2,3,4,5
    

    This would be a very useful feature, and would make bulk deletes more efficient.

    enhancement 
    opened by dantownsend 2
Releases(0.29.0)
  • 0.29.0(Nov 16, 2021)

    Added a __visible_fields filter to PiccoloCRUD. It's a very powerful feature which lets us specify which fields we want the API to return from a GET request (courtesy @sinisaos).

    It can even support joins, but we must supply a max_joins parameter:

    app = PiccoloCRUD(Movie, max_joins=1)
    uvicorn(app)
    

    Then we can do:

    GET /?__visible_fields=id,name,director.name
    

    Which will return:

      {
        "rows": [
            {
                "id": 17,
                "name": "The Hobbit: The Battle of the Five Armies",
                "director": {
                    "name": "Peter Jackson"
                }
            },
            ...
        ]
      }
    

    By specifying exactly which data we want returned, it is much more efficient, especially when fetching large numbers of rows, or with tables with lots of columns.

    Source code(tar.gz)
    Source code(zip)
  • 0.28.1(Nov 11, 2021)

    Fixed a bug with the delete endpoint of PiccoloCRUD. It was returning a 204 response with a body (this isn't allowed, and could cause an exception to be raised in the web server). Thanks to @trondhindenes for reporting this issue.

    Updated Swagger UI to the latest version.

    Source code(tar.gz)
    Source code(zip)
  • 0.28.0(Oct 31, 2021)

  • 0.27.0(Oct 24, 2021)

    You can now pass a schema_extra argument to PiccoloCRUD, which is added to the underlying Pydantic schema.

    This is mostly to support new Piccolo Admin features.

    Source code(tar.gz)
    Source code(zip)
  • 0.26.0(Sep 24, 2021)

  • 0.25.1(Sep 19, 2021)

    • Added examples to CSRF docs (courtesy @sinisaos).
    • Improved SessionAuthBackend - it was too aggressive at rejecting requests when allow_unauthenticated=True (thanks to @Bakz for reporting this).
    Source code(tar.gz)
    Source code(zip)
  • 0.25.0(Sep 7, 2021)

    If you send a GET request to the session_logout endpoint, it will now render a simple logout form. This makes it work much nicer out of the box. Thanks to @sinisaos for adding this.

    Source code(tar.gz)
    Source code(zip)
  • 0.24.1(Sep 1, 2021)

    When using the nested argument in create_pydantic_model, more of the other arguments are passed to the nested models. For example, if include_default_columns is True, both the parent and child models will include their default columns.

    Source code(tar.gz)
    Source code(zip)
  • 0.24.0(Aug 30, 2021)

    Added support for nested models in create_pydantic_model. For each ForeignKey in the Piccolo table, the Pydantic model will contain a sub model for the related table.

    For example:

    class Manager(Table):
        name = Varchar()
    
    class Band(Table):
        name = Varchar()
        manager = ForeignKey(Manager)
    
    BandModel = create_pydantic_model(Band, nested=True)
    

    If we were to write BandModel by hand instead, it would look like this:

    class ManagerModel(BaseModel):
        name: str
    
    class BandModel(BaseModel):
        name: str
        manager: ManagerModel
    

    This feature is designed to work with the new nested output option in Piccolo >= 0.40.0, which returns the data in the correct format to pass directly to the nested Pydantic model.

    band = Band.select(
        Band.id,
        Band.name,
        *Band.manager.all_columns()
    ).first(
    ).output(
        nested=True
    ).run_sync()
    >>> print(band)
    {'id': 1, 'name': 'Pythonistas', 'manager': {'id': 1, 'name': 'Guido'}}
    
    BandModel(**band)
    

    Courtesy @aminalaee.

    Source code(tar.gz)
    Source code(zip)
  • 0.23.1(Aug 13, 2021)

  • 0.23.0(Aug 13, 2021)

    • Fixed MyPy errors (courtesy @sinisaos).
    • Simplification of JWT authentication - it no longer needlessly checks expiry, as PyJWT already does this (courtesy @aminalaee).
    • Substantial increase in code coverage (courtesy @aminalaee and @sinisaos).
    • Increased the minimum PyJWT version, as versions > 2.0.0 return the JWT as a string instead of bytes.
    • Added an option to exclude columns when using create_pydantic_model (courtesy @kucera-lukas).
    Source code(tar.gz)
    Source code(zip)
  • 0.22.0(Aug 6, 2021)

fastapi-crud-sync

Developing and Testing an API with FastAPI and Pytest Syncronous Example Want to use this project? Build the images and run the containers: $ docker-c

null 41 Nov 26, 2021
A rate limiter for Starlette and FastAPI

SlowApi A rate limiting library for Starlette and FastAPI adapted from flask-limiter. Note: this is alpha quality code still, the API may change, and

Laurent Savaete 329 Nov 22, 2021
The template for building scalable web APIs based on FastAPI, Tortoise ORM and other.

FastAPI and Tortoise ORM. Powerful but simple template for web APIs w/ FastAPI (as web framework) and Tortoise-ORM (for working via database without h

prostomarkeloff 73 Nov 18, 2021
Backend, modern REST API for obtaining match and odds data crawled from multiple sites. Using FastAPI, MongoDB as database, Motor as async MongoDB client, Scrapy as crawler and Docker.

Introduction Apiestas is a project composed of a backend powered by the awesome framework FastAPI and a crawler powered by Scrapy. This project has fo

Fran Lozano 45 Nov 19, 2021
Minimal example utilizing fastapi and celery with RabbitMQ for task queue, Redis for celery backend and flower for monitoring the celery tasks.

FastAPI with Celery Minimal example utilizing FastAPI and Celery with RabbitMQ for task queue, Redis for Celery backend and flower for monitoring the

Grega Vrbančič 255 Nov 24, 2021
Social Distancing Detector using deep learning and capable to run on edge AI devices such as NVIDIA Jetson, Google Coral, and more.

Smart Social Distancing Smart Social Distancing Introduction Getting Started Prerequisites Usage Processor Optional Parameters Configuring AWS credent

Neuralet 101 Nov 19, 2021
API & Webapp to answer questions about COVID-19. Using NLP (Question Answering) and trusted data sources.

This open source project serves two purposes. Collection and evaluation of a Question Answering dataset to improve existing QA/search methods - COVID-

deepset 309 Nov 7, 2021
All of the ad-hoc things you're doing to manage incidents today, done for you, and much more!

About What's Dispatch? Put simply, Dispatch is: All of the ad-hoc things you’re doing to manage incidents today, done for you, and a bunch of other th

Netflix, Inc. 3.1k Nov 26, 2021
Example app using FastAPI and JWT

FastAPI-Auth Example app using FastAPI and JWT virtualenv -p python3 venv source venv/bin/activate pip3 install -r requirements.txt mv config.yaml.exa

Sander 19 Mar 28, 2021
Web Inventory tool, takes screenshots of webpages using Pyppeteer (headless Chrome/Chromium) and provides some extra bells & whistles to make life easier.

WitnessMe WitnessMe is primarily a Web Inventory tool inspired by Eyewitness, its also written to be extensible allowing you to create custom function

byt3bl33d3r 539 Nov 22, 2021
Backend Skeleton using FastAPI and Sqlalchemy ORM

Backend API Skeleton Based on @tiangolo's full stack postgres template, with some things added, some things removed, and some things changed. This is

David Montague 13 Nov 8, 2021
FastAPI Admin Dashboard based on FastAPI and Tortoise ORM.

FastAPI ADMIN 中文文档 Introduction FastAPI-Admin is a admin dashboard based on fastapi and tortoise-orm. FastAPI-Admin provide crud feature out-of-the-bo

long2ice 979 Nov 24, 2021
A simple docker-compose app for orchestrating a fastapi application, a celery queue with rabbitmq(broker) and redis(backend)

fastapi - celery - rabbitmq - redis -> Docker A simple docker-compose app for orchestrating a fastapi application, a celery queue with rabbitmq(broker

Kartheekasasanka Kaipa 37 Nov 26, 2021
Ready-to-use and customizable users management for FastAPI

FastAPI Users Ready-to-use and customizable users management for FastAPI Documentation: https://frankie567.github.io/fastapi-users/ Source Code: https

François Voron 1.4k Nov 24, 2021
Analytics service that is part of iter8. Robust analytics and control to unleash cloud-native continuous experimentation.

iter8-analytics iter8 enables statistically robust continuous experimentation of microservices in your CI/CD pipelines. For in-depth information about

null 16 Oct 14, 2021
TODO aplication made with Python's FastAPI framework and Hexagonal Architecture

FastAPI Todolist Description Todolist aplication made with Python's FastAPI framework and Hexagonal Architecture. This is a test repository for the pu

Giovanni Armane 45 Nov 25, 2021
signal-cli-rest-api is a wrapper around signal-cli and allows you to interact with it through http requests

signal-cli-rest-api signal-cli-rest-api is a wrapper around signal-cli and allows you to interact with it through http requests. Features register/ver

Sebastian Noel Lübke 26 Nov 15, 2021
A Python pickling decompiler and static analyzer

Fickling Fickling is a decompiler, static analyzer, and bytecode rewriter for Python pickle object serializations. Pickled Python objects are in fact

Trail of Bits 107 Nov 12, 2021
Install multiple versions of r2 and its plugins via Pip on any system!

r2env This repository contains the tool available via pip to install and manage multiple versions of radare2 and its plugins. r2-tools doesn't conflic

radare org 10 Nov 15, 2021