Intelligent Video Analytics toolkit based on different inference backends.

Overview

English | 中文

OpenIVA

alt OpenIVA

OpenIVA is an end-to-end intelligent video analytics development toolkit based on different inference backends, designed to help individual users and start-ups quickly launch their own video AI services.
OpenIVA implements varied mainstream facial recognition, object detection, segmentation and landmark detection algorithms. And it provides an efficient and lightweight service deployment framework with a modular design. Users only need to replace the algorithm model used for their own tasks.

Features

  1. Common mainstream algorithms
  • Provides latest fast accurate pre-trained models for facial recognition, object detection, segmentation and landmark detection tasks
  1. Multi inference backends
  • Supports TensorlayerX/ TensorRT/ onnxruntime
  1. High performance
  • Achieves high performance on CPU/GPU/Ascend platforms, achieve inference speed above 3000it/s
  1. Asynchronous & multithreading
  • Use multithreading and queue to achieve high device utilization for inference and pre/post-processing
  1. Lightweight service
  • Use Flask for lightweight intelligent application services
  1. Modular design
  • You can quickly start your intelligent analysis service, only need to replace the AI models
  1. GUI visualization tools
  • Start analysis tasks only by clicking buttons, and show visualized results in GUI windows, suitable for multiple tasks

alt Sample Face landmark alt Sample Face recognition alt Sample YOLOX

Performance benchmark

Testing environments

  • i5-10400 6c12t
  • RTX3060
  • Ubuntu18.04
  • CUDA 11.1
  • TensorRT-7.2.3.4
  • onnxruntime with EPs:
    • CPU(Default)
    • CUDA(Manually Compiled)
    • OpenVINO(Manually Compiled)
    • TensorRT(Manually Compiled)

Performance

Facial recognition

Run
python test_landmark.py
batchsize=8, top_k=68, 67 faces in the image

  • Face detection
    Model face_detector_640_dy_sim

    onnxruntime EPs FPS faces per sec
    CPU 32 2075
    OpenVINO 81 5374
    CUDA 105 7074
    TensorRT(FP32) 124 7948
    TensorRT(FP16) 128 8527
  • Face landmark
    Model landmarks_68_pfld_dy_sim

    onnxruntime EPs faces per sec
    CPU 69
    OpenVINO 890
    CUDA 2061
    TensorRT(FP32) 2639
    TensorRT(FP16) 3131

Run
python test_face.py
batchsize=8

  • Face embedding
    Model arc_mbv2_ccrop_sim

    onnxruntime EPs faces per sec
    CPU 212
    OpenVINO 865
    CUDA 1790
    TensorRT(FP32) 2132
    TensorRT(FP16) 2812

Objects detection

Run
python test_yolo.py
batchsize=8 , 4 objects in the image

  • YOLOX objects detect
    Model yolox_s(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 9.3 37.2
    OpenVINO 13 52
    CUDA 77 307
    TensorRT(FP32) 95 380
    TensorRT(FP16) 128 512

    Model yolox_m(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 4 16
    OpenVINO 5.5 22
    CUDA 46.8 187
    TensorRT(FP32) 64 259
    TensorRT(FP16) 119 478

    Model yolox_nano(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 47 188
    OpenVINO 80 320
    CUDA 210 842
    TensorRT(FP32) 244 977
    TensorRT(FP16) 269 1079

    Model yolox_tiny(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 33 133
    OpenVINO 43 175
    CUDA 209 839
    TensorRT(FP32) 248 995
    TensorRT(FP16) 327 1310

Progress

  • Multi inference backends

    • onnxruntime
      • CPU
      • CUDA
      • TensorRT
      • OpenVINO
    • TensorlayerX
    • TensorRT
  • Asynchronous & multithreading

    • Data generate threads
    • AI compute threads
    • Multifunctional threads
    • Collecting threads
  • Lightweight service

    • prototype
  • GUI visualization tools

  • Common algorithms

    • Facial recognition

      • Face detection

      • Face landmark

      • Face embedding

    • Object detection

      • YOLOX
    • Semantic/Instance segmentation

    • Scene classification

      • prototype
  • Data I/O

    • Video decoding
      • OpenCV decoding
        • Local video files
        • Network stream videos
    • Data management
      • Facial identity database
      • Data serialization
You might also like...
A repository built on the Flow software package to explore cyber-security attacks on intelligent transportation systems.

A repository built on the Flow software package to explore cyber-security attacks on intelligent transportation systems.

RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems

RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems This is our implementation for the paper: Weibo Gao, Qi Liu*, Zhenya Hu

A privacy-focused, intelligent security camera system.
A privacy-focused, intelligent security camera system.

Self-Hosted Home Security Camera System A privacy-focused, intelligent security camera system. Features: Multi-camera support w/ minimal configuration

This is a repository for a Semantic Segmentation inference API using the Gluoncv CV toolkit
This is a repository for a Semantic Segmentation inference API using the Gluoncv CV toolkit

BMW Semantic Segmentation GPU/CPU Inference API This is a repository for a Semantic Segmentation inference API using the Gluoncv CV toolkit. The train

This is a repository for a semantic segmentation inference API using the OpenVINO toolkit
This is a repository for a semantic segmentation inference API using the OpenVINO toolkit

BMW-IntelOpenVINO-Segmentation-Inference-API This is a repository for a semantic segmentation inference API using the OpenVINO toolkit. It's supported

CausalNLP is a practical toolkit for causal inference with text as treatment, outcome, or "controlled-for" variable.

CausalNLP CausalNLP is a practical toolkit for causal inference with text as treatment, outcome, or "controlled-for" variable. Install pip install -U

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

Note: This is an alpha (preview) version which is still under refining. nn-Meter is a novel and efficient system to accurately predict the inference l

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Owner
Quantum Liu
RAmen
Quantum Liu
Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

Numenta 6.2k Feb 12, 2021
A platform for intelligent agent learning based on a 3D open-world FPS game developed by Inspir.AI.

Wilderness Scavenger: 3D Open-World FPS Game AI Challenge This is a platform for intelligent agent learning based on a 3D open-world FPS game develope

null 46 Nov 24, 2022
Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready inference.

Yolov5 running on TorchServe (GPU compatible) ! This is a dockerfile to run TorchServe for Yolo v5 object detection model. (TorchServe (PyTorch librar

null 82 Nov 29, 2022
Monocular 3D pose estimation. OpenVINO. CPU inference or iGPU (OpenCL) inference.

human-pose-estimation-3d-python-cpp RealSenseD435 (RGB) 480x640 + CPU Corei9 45 FPS (Depth is not used) 1. Run 1-1. RealSenseD435 (RGB) 480x640 + CPU

Katsuya Hyodo 8 Oct 3, 2022
Data-depth-inference - Data depth inference with python

Welcome! This readme will guide you through the use of the code in this reposito

Marco 3 Feb 8, 2022
PyTorch implementation of the end-to-end coreference resolution model with different higher-order inference methods.

End-to-End Coreference Resolution with Different Higher-Order Inference Methods This repository contains the implementation of the paper: Revealing th

Liyan 52 Jan 4, 2023
A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Awesome Pretrained StyleGAN2 A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution. Note the readme is a

Justin 1.1k Dec 24, 2022
House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent for Professional Architects

House-GAN++ Code and instructions for our paper: House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent

null 122 Dec 28, 2022
An intelligent, flexible grammar of machine learning.

An english representation of machine learning. Modify what you want, let us handle the rest. Overview Nylon is a python library that lets you customiz

Palash Shah 79 Dec 2, 2022
CityLearn Challenge Multi-Agent Reinforcement Learning for Intelligent Energy Management, 2020, PikaPika team

Citylearn Challenge This is the PyTorch implementation for PikaPika team, CityLearn Challenge Multi-Agent Reinforcement Learning for Intelligent Energ

bigAIdream projects 10 Oct 10, 2022