The deployment framework aims to provide a simple, lightweight, fast integrated, pipelined deployment framework that ensures reliability, high concurrency and scalability of services.

Overview

Savior

save your time.

只在Ubuntu18.04下完成全部测试,其他平台暂时未测试。

目前项目还处于早期开发阶段,如有任何问题,欢迎添加微信nsnovio,备注部署,进群交流。

背景

savior是一个能够进行快速集成算法模块并支持高性能部署的轻量开发框架。能够帮助将团队进行快速想法验证(PoC),避免重复的去github上找模型然后复现模型;能够帮助团队将功能进行流程拆解,很方便的提高分布式执行效率;能够有效减少代码冗余,减少不必要负担。

workflow的已经做好的轮子很多,例如perfectpolyaxondagster等。 之所以开发一个新的,主要原因是那些框架都太heavy了,对于大部分用户来说没法直接白嫖。

这个项目的核心目的就是能够减少大家的重复性开发,能够把绝大部分能够直接白嫖的东西放在框架里面,然后大家专注于自己的业务属性上,提升大家的工作效率。

特性

  1. 弹性伸缩:用户可以根据目前的请求量,自定义配置机器数。方便项目上量,并且保证服务器资源吃满(支持K8S)。
  2. 流程DAG:用户通过自定义自己的流程,框架支持DAG,保证流程的最高的并行度。
  3. 容灾能力强:集群中所有节点都是相同作用,不会因为部分节点挂掉而服务崩溃。
  4. 可扩展性强:框架主要是实现了一种设计模式,开发者只需要按照当前设计模式,扩展性无上限。
  5. 部署便捷:部署到上线不会超过5分钟(不考虑网速)。

依赖的第三方组件

  • rabbitmq:用于celery进行分布式的任务分发

  • triton:用于gpu端的模型服务的集中部署

  • milvus:用于特征向量搜索,存储【推荐有搜索需求的用户自行配置】

    如果觉得milvus太大,用户可以根据自己的自身情况直接使用faiss或者nmslib。并且自己实现对应helper。

框架中已集成的算法

更多开源模型欢迎在issue中补充,也十分欢迎您的PR。

人脸相关

OCR相关

  • DB 文本检测
  • CRNN 文本识别
  • 版式分析
  • 文本图像方向矫正
  • 文本方向检测
  • 常见扇形环形转换为矩形(针对于segmentation base的检测方案)

图像搜索

通用

  • NRIQA

官方已适配模型下载地址(不定时更新):

根据自己的需要下载模型,不用全部下载。

简单使用教程

  1. 克隆项目git clone https://github.com/novioleo/Savior.git到本地。或者下载release下面的source包。
  2. 启动rabbitmq,推荐使用docker启动:docker run --restart=always -d --hostname celery-broker --name celery-broker -p5672:5672 -p15672:15672 -e RABBITMQ_DEFAULT_USER=guest -e RABBITMQ_DEFAULT_PASS=guest rabbitmq:3-management
  3. 启动triton,推荐使用docker(需要安装nvidia-docker)启动:docker run --gpus=all --name=triton-server -p8000:8000 -p8001:8001 -v/path/to/your/model/repo/path:/models nvcr.io/nvidia/tritonserver:20.12-py3 tritonserver --model-repository=/models,其中/path/to/your/model/repo/path是网盘中triton文件夹下载的所在文件夹。
  4. 修改项目配置,进入Savior文件夹中,进入Deployment包中,复制server_config.py.template并重命名为server_config.py,修改里面triton、rabbitmq的配置。
  5. 配置python与安装依赖,通过控制台进入Savior文件夹中,创建环境:conda create -n SaviorEnv python=3.8,激活环境source activate SaviorEnv,安装依赖:python -m pip install nvidia-pyindex==1.0.6 && python -m pip install -r requirements.txt
  6. 启动ConsumerWorker,通过控制台进入Savior文件夹中,启动worker:celery -A Deployment.ConsumerWorker worker --loglevel=INFO,如果一切配置正确会显示已经成功加载Task。
  7. 启动DispatchServer,通过控制台进入Savior文件夹中,启动server:python Deployment/DispathServer.py,启动成功会看到端口信息等。
  8. 测试接口服务,推荐使用apifox进行接口调用测试,可以通过post请求测试ocr_interface/general_ocr接口,传入参数image_url,发送请求(第一次运行需要等待,模型需要预热,五次之后基本上时间会稳定),会得到一个OSS的路径,如果OSS使用的是Dummy(默认),则找到/tmp/DummyOSS-temp-directory/{bucket_name}/{path}对应的文件。

生产级使用教程点我

接口结果预览

OCR相关

自然场景下OCR

如何在自有项目下开发?

移步至:DevelopTutorial

感谢

感谢各位开源项目大佬的无私奉献。

You might also like...
Official PyTorch implementation of "AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks"

AASIST This repository provides the overall framework for training and evaluating audio anti-spoofing systems proposed in 'AASIST: Audio Anti-Spoofing

A lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look At CoefficienTs)
A lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look At CoefficienTs)

Real-time Instance Segmentation and Lane Detection This is a lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two
Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

512x512 flowers after 12 hours of training, 1 gpu 256x256 flowers after 12 hours of training, 1 gpu Pizza 'Lightweight' GAN Implementation of 'lightwe

This is an official pytorch implementation of Lite-HRNet: A Lightweight High-Resolution Network.
This is an official pytorch implementation of Lite-HRNet: A Lightweight High-Resolution Network.

Lite-HRNet: A Lightweight High-Resolution Network Introduction This is an official pytorch implementation of Lite-HRNet: A Lightweight High-Resolution

Lite-HRNet: A Lightweight High-Resolution Network

LiteHRNet Benchmark 🔥 🔥 Based on MMsegmentation 🔥 🔥 Cityscapes FCN resize concat config mIoU last mAcc last eval last mIoU best mAcc best eval bes

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

Owner
Tao Luo
Algorithmer.
Tao Luo
This project aims to explore the deployment of Swin-Transformer based on TensorRT, including the test results of FP16 and INT8.

Swin Transformer This project aims to explore the deployment of SwinTransformer based on TensorRT, including the test results of FP16 and INT8. Introd

maggiez 87 Dec 21, 2022
Examples of using f2py to get high-speed Fortran integrated with Python easily

f2py Examples Simple examples of using f2py to get high-speed Fortran integrated with Python easily. These examples are also useful to troubleshoot pr

Michael 35 Aug 21, 2022
torchlm is aims to build a high level pipeline for face landmarks detection, it supports training, evaluating, exporting, inference(Python/C++) and 100+ data augmentations

??A high level pipeline for face landmarks detection, supports training, evaluating, exporting, inference and 100+ data augmentations, compatible with torchvision and albumentations, can easily install with pip.

DefTruth 142 Dec 25, 2022
banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's applied reinforcement learning platform, Reagent.

Bandit ML 51 Dec 22, 2022
A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

A Light and Fast Face Detector for Edge Devices Big News: LFD, which is a big update of LFFD, now is released (2021.03.09). It is strongly recommended

YonghaoHe 1.3k Dec 25, 2022
ivadomed is an integrated framework for medical image analysis with deep learning.

Repository on the collaborative IVADO medical imaging project between the Mila and NeuroPoly labs.

null 144 Dec 19, 2022
Lightweight mmm - Lightweight (Bayesian) Media Mix Model

Lightweight (Bayesian) Media Mix Model This is not an official Google product. L

Google 342 Jan 3, 2023
PyDeepFakeDet is an integrated and scalable tool for Deepfake detection.

PyDeepFakeDet An integrated and scalable library for Deepfake detection research. Introduction PyDeepFakeDet is an integrated and scalable Deepfake de

Junke, Wang 49 Dec 11, 2022
Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021)

Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021) Overview of paths used in DIG and IG. w is the word being attributed. The

INK Lab @ USC 17 Oct 27, 2022