English | 中文
OpenIVA
OpenIVA is an end-to-end intelligent video analytics development toolkit based on different inference backends, designed to help individual users and start-ups quickly launch their own video AI services.
OpenIVA implements varied mainstream facial recognition, object detection, segmentation and landmark detection algorithms. And it provides an efficient and lightweight service deployment framework with a modular design. Users only need to replace the algorithm model used for their own tasks.
Features
- Common mainstream algorithms
- Provides latest fast accurate pre-trained models for facial recognition, object detection, segmentation and landmark detection tasks
- Multi inference backends
- Supports TensorlayerX/ TensorRT/ onnxruntime
- High performance
- Achieves high performance on CPU/GPU/Ascend platforms, achieve inference speed above 3000it/s
- Asynchronous & multithreading
- Use multithreading and queue to achieve high device utilization for inference and pre/post-processing
- Lightweight service
- Use Flask for lightweight intelligent application services
- Modular design
- You can quickly start your intelligent analysis service, only need to replace the AI models
- GUI visualization tools
- Start analysis tasks only by clicking buttons, and show visualized results in GUI windows, suitable for multiple tasks
Performance benchmark
Testing environments
- i5-10400 6c12t
- RTX3060
- Ubuntu18.04
- CUDA 11.1
- TensorRT-7.2.3.4
- onnxruntime with EPs:
- CPU(Default)
- CUDA(Manually Compiled)
- OpenVINO(Manually Compiled)
- TensorRT(Manually Compiled)
Performance
Facial recognition
Run
python test_landmark.py
batchsize=8
, top_k=68
, 67 faces in the image
-
Face detection
Modelface_detector_640_dy_sim
onnxruntime EPs FPS faces per sec CPU 32 2075 OpenVINO 81 5374 CUDA 105 7074 TensorRT(FP32) 124 7948 TensorRT(FP16) 128 8527 -
Face landmark
Modellandmarks_68_pfld_dy_sim
onnxruntime EPs faces per sec CPU 69 OpenVINO 890 CUDA 2061 TensorRT(FP32) 2639 TensorRT(FP16) 3131
Run
python test_face.py
batchsize=8
-
Face embedding
Modelarc_mbv2_ccrop_sim
onnxruntime EPs faces per sec CPU 212 OpenVINO 865 CUDA 1790 TensorRT(FP32) 2132 TensorRT(FP16) 2812
Objects detection
Run
python test_yolo.py
batchsize=8
, 4 objects in the image
-
YOLOX objects detect
Modelyolox_s(ms_coco)
onnxruntime EPs FPS Objects per sec CPU 9.3 37.2 OpenVINO 13 52 CUDA 77 307 TensorRT(FP32) 95 380 TensorRT(FP16) 128 512 Model
yolox_m(ms_coco)
onnxruntime EPs FPS Objects per sec CPU 4 16 OpenVINO 5.5 22 CUDA 46.8 187 TensorRT(FP32) 64 259 TensorRT(FP16) 119 478 Model
yolox_nano(ms_coco)
onnxruntime EPs FPS Objects per sec CPU 47 188 OpenVINO 80 320 CUDA 210 842 TensorRT(FP32) 244 977 TensorRT(FP16) 269 1079 Model
yolox_tiny(ms_coco)
onnxruntime EPs FPS Objects per sec CPU 33 133 OpenVINO 43 175 CUDA 209 839 TensorRT(FP32) 248 995 TensorRT(FP16) 327 1310
Progress
-
Multi inference backends
- onnxruntime
- CPU
- CUDA
- TensorRT
- OpenVINO
- TensorlayerX
- TensorRT
- onnxruntime
-
Asynchronous & multithreading
- Data generate threads
- AI compute threads
- Multifunctional threads
- Collecting threads
-
Lightweight service
- prototype
-
GUI visualization tools
-
Common algorithms
-
Facial recognition
-
Face detection
-
Face landmark
-
Face embedding
-
-
Object detection
- YOLOX
-
Semantic/Instance segmentation
-
Scene classification
- prototype
-
-
Data I/O
- Video decoding
- OpenCV decoding
- Local video files
- Network stream videos
- OpenCV decoding
- Data management
- Facial identity database
- Data serialization
- Video decoding