Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Last update: Dec 27, 2022

Overview

Ultralight-SimplePose

Support NCNN mobile terminal deployment
Based on MXNET(>=1.5.1) GLUON(>=0.7.0) framework
Top-down strategy: The input image is the person ROI detected by the object detector
Lightweight mobile terminal human body posture key point model(COCO 17 person_keypoints)
Detector:https://github.com/dog-qiuqiu/MobileNetv2-YOLOV3

Model

Mobile inference frameworks benchmark (4*ARM_CPU)

Network	Resolution	Inference time (NCNN/Kirin 990)	FLOPS	Weight size	HeatmapAccuracy
Ultralight-Nano-SimplePose	W:192 H:256	~5.4ms	0.224BFlops	2.3MB	74.3%

COCO2017 val keypoints metrics evaluate

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.518
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.816
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.558
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.498
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.549
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 20 ] = 0.563
 Average Recall     (AR) @[ IoU=0.50      | area=   all | maxDets= 20 ] = 0.837
 Average Recall     (AR) @[ IoU=0.75      | area=   all | maxDets= 20 ] = 0.607
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.535
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.604

Install

pip install mxnet-cu101 gluoncv
pip install opencv-python cython pycocotools

Install mxnet according to your own cuda version

Demo

Test picture

python img_demo.py

Test camera stream

python cam_demo

How To Train

Download the coco2017 dataset

http://images.cocodataset.org/zips/train2017.zip
http://images.cocodataset.org/annotations/annotations_trainval2017.zip
http://images.cocodataset.org/zips/val2017.zip
Unzip the downloaded dataset zip file to the coco directory
交流qq群:1062122604

Train

python train_simple_pose.py

Ncnn Deploy

Dependent library: Opencv Ncnn
Read the camera video stream test by default, if you test the picture, please modify the code

Install ncnn

$ git clone https://github.com/Tencent/ncnn.git
$ cd <ncnn-root-dir>
$ mkdir -p build
$ cd build
$ make -j4
$ make install

Run ncnn sample

$ cp -rf ncnn/build/install/include ./Ultralight-SimplePose/ncnnsample/
$ cp -rf ncnn/build/install/lib ./Ultralight-SimplePose/ncnnsample/
$ g++ -o ncnnpose ncnnpose.cpp -I include/ncnn/ lib/libncnn.a `pkg-config --libs --cflags opencv` -fopenmp
$ ./ncnnpose

Ncnn Picture test results

Android sample

https://github.com/WZTENG/YOLOv5_NCNN

Thanks

SimplePose Paper:https://arxiv.org/abs/1804.06208
https://github.com/Tencent/ncnn
https://gluon-cv.mxnet.io/

You might also like...

Img-process-manual - Utilize Python Numpy and Matplotlib to realize OpenCV baisc image processing function

Img-process-manual - Opencv Library basic graphic processing algorithm coding reproduction based on Numpy and Matplotlib library

2 Dec 12, 2022

thundernet ncnn

MMDetection_Lite 基于mmdetection 实现一些轻量级检测模型，安装方式和mmdeteciton相同 voc0712 voc 0712训练 voc2007测试 coco预训练 thundernet_voc_shufflenetv2_1.5 input shape mAP 320

39 Dec 5, 2022

quantize aware training package for NCNN on pytorch

ncnnqat ncnnqat is a quantize aware training package for NCNN on pytorch. Table of Contents ncnnqat Table of Contents Installation Usage Code Examples

62 Nov 23, 2022

A high-performance anchor-free YOLO. Exceeding yolov3~v5 with ONNX, TensorRT, NCNN, and Openvino supported.

YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and industrial communities. For more details, please refer to our report on Arxiv.

7.7k Jan 6, 2023

YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with ONNX, TensorRT, ncnn, and OpenVINO supported.

Introduction YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and ind

7.7k Jan 3, 2023

NFT-Price-Prediction-CNN - Using visual feature extraction, prices of NFTs are predicted via CNN (Alexnet and Resnet) architectures.

5 Nov 3, 2022

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Comments

使用onnxruntime加载onnx模型报错

（1）INVALID_ARGUMENT : Non-zero status code returned while running BatchNormalization node. （2）Status Message: Invalid input scale: NumDimensions() != 3 参考：https://github.com/onnx/models/issues/156解决原因：mxnet转化为onnx时，产生错误的模型，BatchNormalization(space=0)

opened by JayL323 1
How to use Yolov3-Nano for object detection instead of yolo3_mobilenet1.0_coco

Hi,

In the current code you are using "yolo3_mobilenet1.0_coco" for object detection. Can you please tell me how to use "Yolov3-Nano"( https://github.com/dog-qiuqiu/MobileNet-Yolo/tree/master/MobileNetV2-YOLOv3-Nano ) for object detection in the code instead of "yolo3_mobilenet1.0_coco".

Thank you @dog-qiuqiu

opened by sakethkhandavalli 1

Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Related tags

Overview

Ultralight-SimplePose

Model

Mobile inference frameworks benchmark (4*ARM_CPU)

COCO2017 val keypoints metrics evaluate

Install

Demo

Test picture

Test camera stream

How To Train

Download the coco2017 dataset

Train

Ncnn Deploy

Install ncnn

Run ncnn sample

Ncnn Picture test results

Android sample

Thanks

You might also like...

Img-process-manual - Utilize Python Numpy and Matplotlib to realize OpenCV baisc image processing function

thundernet ncnn

quantize aware training package for NCNN on pytorch

A high-performance anchor-free YOLO. Exceeding yolov3~v5 with ONNX, TensorRT, NCNN, and Openvino supported.

YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with ONNX, TensorRT, ncnn, and OpenVINO supported.

NFT-Price-Prediction-CNN - Using visual feature extraction, prices of NFTs are predicted via CNN (Alexnet and Resnet) architectures.

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Example scripts for the detection of lanes using the ultra fast lane detection model in Tensorflow Lite.

Lane assist for ETS2, built with the ultra-fast-lane-detection model.

Comments

使用onnxruntime加载onnx模型报错

How to use Yolov3-Nano for object detection instead of yolo3_mobilenet1.0_coco

Owner

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

CenterFace(size of 7.3MB) is a practical anchor-free face detection and alignment method for edge devices.

Weakly Supervised Posture Mining with Reverse Cross-entropy for Fine-grained Classification

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

Full body anonymization - Realistic Full-Body Anonymization with Surface-Guided GANs

WormMovementSimulation - 3D Simulation of Worm Body Movement with Neurons attached to its body

Based on Yolo's low-power, ultra-lightweight universal target detection algorithm, the parameter is only 250k, and the speed of the smart phone mobile terminal can reach ~300fps+

SMPL-X: A new joint 3D model of the human body, face and hands together

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles