Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Overview

ONNX-Mobile-Human-Pose-3D

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model.

Mobile Human 3D Pose mation ONNX Original image for inference: (https://static2.diariovasco.com/www/pre2017/multimedia/noticias/201412/01/media/DF0N5391.jpg)

⚠️ Known issues

  • The models works well when the person is looking forward and without occlusions, it will start to fail as soon as the person is occluded.
  • The model is fast, but the 3D representation is slow due to matplotlib, this will be fixed. The 3d representation can be ommitted for faster inference by setting draw_3dpose to False

Requirements

  • OpenCV, imread-from-url, scipy, onnx and onnxruntime. Also, pafy and youtube-dl are required for youtube video inference.

Installation

pip install -r requirements.txt
pip install pafy youtube-dl

ONNX model

The original models were converted to different formats (including .onnx) by PINTO0309, download the models from his repository and save them into the models folder.

  • YOLOv5s: You will also need an object detector to first detect the people in the image. Download the model from the model zoo and save the .onnx version into the models folder.

Original model

The original model was taken from the original repository.

Examples

  • Image inference:
python imagePoseEstimation.py 
  • Video inference:
python videoPoseEstimation.py
  • Webcam inference:
python webcamPoseEstimation.py

Inference video Example

Mobile Human 3D Pose mation ONNX

References:

You might also like...
Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite.
Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite.

TFLite-MobileStereoNet Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite. Stereo depth estimati

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX.
Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX.

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX. The repository combines a class agnostic object localizer to first detect the objects in the image, and next a ResNet50 model trained on ImageNet is used to label each box.

Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX

ONNX msg_chn_wacv20 depth completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20 model in

This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).
This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).

Github Code of "MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices" Introduction This repo is official PyTorch implementatio

An executor that loads ONNX models and embeds documents using the ONNX runtime.

ONNXEncoder An executor that loads ONNX models and embeds documents using the ONNX runtime. Usage via Docker image (recommended) from jina import Flow

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.
Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS.

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS. It currently supports four examples for you to quickly experience the power of ONNX Runtime Web.

A repository that shares tuning results of trained models generated by TensorFlow / Keras. Post-training quantization (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization), Quantization-aware training. TensorFlow Lite. OpenVINO. CoreML. TensorFlow.js. TF-TRT. MediaPipe. ONNX. [.tflite,.h5,.pb,saved_model,tfjs,tftrt,mlmodel,.xml/.bin, .onnx] A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want.
A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want.

sne4onnx A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or

Comments
  • [Question] What kind of idea do you base the estimation of depth

    [Question] What kind of idea do you base the estimation of depth

    Thank you for sharing. Could you tell me the idea behind the following line? https://github.com/ibaiGorordo/ONNX-Mobile-Human-Pose-3D/blob/main/mobileHumanPose/mobileHumanPose.py#L161

    I understand you need abs_depth. For instance, using the average height will be one of solutions. focal_length(fixed value) : abs_depth(unknown value) = height_in_pixel(known value) : average_height(fixed value) Using the above equation, we can get abs_depth. (It may not work well, though)

    In your code, it looks you use area, and add +500. The values may be determined by experiments, but could you tell me the basic idea of this?

    opened by iwatake2222 2
  • Inference problem

    Inference problem

    Hi,thanks for the great work. I follow your Readme and download the required models including Mobile Human Pose and YOLOv5s, and below is the inference result and which is not as accurate as i expect. The input image size is 512*512, and i have no idea to choose the proper detection model(different resolution),so i just tried it all,which not improved the result.Could you point out what the problem is?

    output

    opened by SienChen1997 2
  • Error downloading model file(mobile_human_pose_working_well_256x256.onnx)?

    Error downloading model file(mobile_human_pose_working_well_256x256.onnx)?

    I can't download through the link(https://github.com/PINTO0309/PINTO_model_zoo/blob/main/135_CoEx/download.sh).The error screenshot is as follows: 09d4da6e7f4c29ed57d53742fc38d16 ,I can not find the cookie file.Can you provide an absolute download link,thank you.

    opened by shnulailai 2
  • How to improve the quality of keypoints 2D and 3D?

    How to improve the quality of keypoints 2D and 3D?

    I have tested on rtsp live feed to get 2D and 3D poses. but i see so much variation on keypoints extracted due to which even 2D visualization is not looking good. How can we make it better, is there any better model file available. and what is role of Focal length and principal points in computation because i see they are commented in pix2cam function.

    opened by chandelarvind 0
Owner
Ibai Gorordo
Passionate about sensors, technology and their potential to help people.
Ibai Gorordo
ONNX-PackNet-SfM: Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Ibai Gorordo 13 Oct 5, 2022
Python scripts form performing stereo depth estimation using the CoEx model in ONNX.

ONNX-CoEx-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the CoEx model in ONNX. Stereo depth estimation on the

Ibai Gorordo 7 Oct 31, 2022
Python scripts form performing stereo depth estimation using the HITNET model in ONNX.

ONNX-HITNET-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in ONNX. Stereo depth estimation on

Ibai Gorordo 30 Nov 8, 2022
Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

ONNX-MobileStereoNet Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX Stereo depth estimation on the cone

Ibai Gorordo 22 Oct 18, 2022
Python scripts for performing lane detection using the LSTR model in ONNX

ONNX LSTR Lane Detection Python scripts for performing lane detection using the Lane Shape Prediction with Transformers (LSTR) model in ONNX. Requirem

Ibai Gorordo 29 Aug 30, 2022
Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONNX.

ONNX-HybridNets-Multitask-Road-Detection Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONN

Ibai Gorordo 44 Nov 22, 2022
Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

ONNX Object Localization Network Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX. Ori

Ibai Gorordo 15 Oct 14, 2022
Python scripts for performing stereo depth estimation using the HITNET Tensorflow model.

HITNET-Stereo-Depth-estimation Python scripts for performing stereo depth estimation using the HITNET Tensorflow model from Google Research. Stereo de

Ibai Gorordo 72 Nov 10, 2022
Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch .

PyTorch-High-Res-Stereo-Depth-Estimation Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch. Stereo dep

Ibai Gorordo 26 Nov 24, 2022
Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite.

TFLite-HITNET-Stereo-depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite. Stereo depth e

Ibai Gorordo 22 Oct 20, 2022