A RESTful API for creating and monitoring resource components of a hypothetical build system. Built with FastAPI and pydantic. Complete with testing and CI.

Overview

diskspace-monitor-CRUD

Diskspace Monitor Test Suite

Background

The build system is part of a large environment with a multitude of different components. Many of the components have some sort of storage (examples: crash dump handler, versioning system, build distribution). To ensure none of the services go down due to a lack of available storage, the systems have an agent that reports disk usage back to a central monitoring facility, which evaluates the collected data against preset rules and provides status and warnings through API endpoints.

The current project implements this central monitoring facility.

To read API Documentation, see API_Documentation.md.

Getting Started Without Docker

Prerequisites

Python >= 3.8 and pip are the only prerequisite. I personally use Pipenv but have provided a requirements.txt file for pip.

pip install --upgrade pip

Installation

  1. Clone the repo
git clone  https://github.com/NHopewell/diskspace-monitor-CRUD

cd diskspace-monitor-CRUD
  1. Create a virtual environment of your choice (in this example, venv)
python -m venv env
  1. Activate the virtual environment
source env/bin/activate
  1. Install the source package in the virtual environment
pip install -e .
  1. Install requirements in virtual environment. If you would like to run tests and add onto the project, install the requirements_dev file instead.
# prod requirments
pip install -r requirements.txt

# dev requirements
pip install -r requirements_dev.txt

Usage

The application code which powers the API can be found in src/diskspacemonitor/. To run the webserver:

cd src/diskspacemonitor/

uvicorn main:app --reload

Now our monitoring system is being served over localhost. You can run my test script which automates sending requests to each end point:

python scripts/example_automated_api_calls.py

This script posts some system components and events (some of which triggered warnings in the system), we can also curl these endpoints to see:

# events
curl http://127.0.0.1:8000/v1/component_events | python -m json.tool

# warnings triggered
curl http://127.0.0.1:8000/v1/resource_warnings | python -m json.tool

To see documentation auto-generated by FastAPI, go to: http://127.0.0.1:8000/docs

Getting Started With Docker

  1. Clone the repo
git clone  https://github.com/NHopewell/diskspace-monitor-CRUD

cd diskspace-monitor-CRUD
  1. Build the Dockerfile:
docker build . -t diskspace-monitor
  1. You'll notice in the Dockerfile that we are using the port 8000. Now run the Docker image with port forwarding:
docker run -p 8000:8000 diskspace-monitor

The application is now accessible over localhost http://127.0.0.1:8000/docs

Testing and CI

This project is setup with the following things to ensure PEP8 compliance and proper builds:

  • pre-commit hooks including black, Flake8, and other hooks.
  • local tests for both the API and underlying models with Pytest.
  • virtual env management with tox to run pytests on different Python versions and environments.
  • github actions to automatically run tox with these different Python versions across different operating systems when changes are made to the repo.

To execute all tests manually in your virtualenv, run:

pytest

To execute all tests on your system in multiple virtual environments with different configurations, run:

tox

This will run the test suite in 6 different virtural environments using ubuntu and Windows, each with Python versions 3.8, 3.9. and 3.10.

On git pushes to master or pull requests, tox will be run in these 6 environments concurrently on separate machines.

You might also like...
Sample-fastapi - A sample app using Fastapi that you can deploy on App Platform

Getting Started We provide a sample app using Fastapi that you can deploy on App

Flask-vs-FastAPI - Understanding Flask vs FastAPI Web Framework. A comparison of two different RestAPI frameworks.
Flask-vs-FastAPI - Understanding Flask vs FastAPI Web Framework. A comparison of two different RestAPI frameworks.

Flask-vs-FastAPI Understanding Flask vs FastAPI Web Framework. A comparison of two different RestAPI frameworks. IntroductionIn Flask is a popular mic

FastAPI Server Session is a dependency-based extension for FastAPI that adds support for server-sided session management

FastAPI Server-sided Session FastAPI Server Session is a dependency-based extension for FastAPI that adds support for server-sided session management.

Fastapi-ml-template - Fastapi ml template with python

FastAPI ML Template Run Web API Local $ sh run.sh # poetry run uvicorn app.mai

Stac-fastapi built on Tile38 and Redis to support caching

stac-fastapi-caching Stac-fastapi built on Tile38 to support caching. This code is built on top of stac-fastapi-elasticsearch 0.1.0 with pyle38, a Pyt

Cookiecutter API for creating Custom Skills for Azure Search using Python and Docker

cookiecutter-spacy-fastapi Python cookiecutter API for quick deployments of spaCy models with FastAPI Azure Search The API interface is compatible wit

🍃 A comprehensive monitoring and alerting solution for the status of your Chia farmer and harvesters.
🍃 A comprehensive monitoring and alerting solution for the status of your Chia farmer and harvesters.

chia-monitor A monitoring tool to collect all important metrics from your Chia farming node and connected harvesters. It can send you push notificatio

High-performance Async REST API, in Python. FastAPI + GINO + Arq + Uvicorn (w/ Redis and PostgreSQL).

fastapi-gino-arq-uvicorn High-performance Async REST API, in Python. FastAPI + GINO + Arq + Uvicorn (powered by Redis & PostgreSQL). Contents Get Star

Owner
Nick Hopewell
Nick Hopewell
Restful Api developed with Flask using Prometheus and Grafana for monitoring and containerization with Docker :rocket:

Hephaestus ?? In Greek mythology, Hephaestus was either the son of Zeus and Hera or he was Hera's parthenogenous child. ... As a smithing god, Hephaes

Yasser Tahiri 16 Oct 7, 2022
Fastapi performans monitoring

Fastapi-performans-monitoring This project is a simple performance monitoring for FastAPI. License This project is licensed under the terms of the MIT

bilal alpaslan 11 Dec 31, 2022
Simple FastAPI Example : Blog API using FastAPI : Beginner Friendly

fastapi_blog FastAPI : Simple Blog API with CRUD operation Steps to run the project: git clone https://github.com/mrAvi07/fastapi_blog.git cd fastapi-

Avinash Alanjkar 1 Oct 8, 2022
OpenAPI for Todolist RESTful API

swagger-client OpenAPI for Todolist RESTful API This Python package is automatically generated by the Swagger Codegen project: API version: 1 Package

Iko Afianando 1 Dec 19, 2021
flask extension for integration with the awesome pydantic package

flask extension for integration with the awesome pydantic package

null 249 Jan 6, 2023
Hyperlinks for pydantic models

Hyperlinks for pydantic models In a typical web application relationships between resources are modeled by primary and foreign keys in a database (int

Jaakko Moisio 10 Apr 18, 2022
FastAPI-Amis-Admin is a high-performance, efficient and easily extensible FastAPI admin framework. Inspired by django-admin, and has as many powerful functions as django-admin.

简体中文 | English 项目介绍 FastAPI-Amis-Admin fastapi-amis-admin是一个拥有高性能,高效率,易拓展的fastapi管理后台框架. 启发自Django-Admin,并且拥有不逊色于Django-Admin的强大功能. 源码 · 在线演示 · 文档 · 文

AmisAdmin 318 Dec 31, 2022
fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability

FastAPI2 Admin Introduction fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability. Now

Glib 14 Dec 5, 2022
:rocket: CLI tool for FastAPI. Generating new FastAPI projects & boilerplates made easy.

Project generator and manager for FastAPI. Source Code: View it on Github Features ?? Creates customizable project boilerplate. Creates customizable a

Yagiz Degirmenci 1k Jan 2, 2023
Пример использования GraphQL Ariadne с FastAPI и сравнение его с GraphQL Graphene FastAPI

FastAPI Ariadne Example Пример использования GraphQL Ariadne с FastAPI и сравнение его с GraphQL Graphene FastAPI - GitHub ###Запуск на локальном окру

ZeBrains Team 9 Nov 10, 2022