Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Related tags

Deep Learning pose-estimation onnx 3d-pose-estimation onnxruntime yolov5s
Overview

ONNX-Mobile-Human-Pose-3D

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model.

Mobile Human 3D Pose mation ONNX Original image for inference: (https://static2.diariovasco.com/www/pre2017/multimedia/noticias/201412/01/media/DF0N5391.jpg)

❗ ⚠️ Known issues

  • The models works well when the person is looking forward and without occlusions, it will start to fail as soon as the person is occluded.
  • The model is fast, but the 3D representation is slow due to matplotlib, this will be fixed. The 3d representation can be ommitted for faster inference by setting draw_3dpose to False

Requirements

  • OpenCV, imread-from-url, scipy, onnx and onnxruntime. Also, pafy and youtube-dl are required for youtube video inference.

Installation

pip install -r requirements.txt
pip install pafy youtube-dl

ONNX model

The original models were converted to different formats (including .onnx) by PINTO0309, download the models from his repository and save them into the models folder.

  • YOLOv5s: You will also need an object detector to first detect the people in the image. Download the model from the model zoo and save the .onnx version into the models folder.

Original model

The original model was taken from the original repository.

Examples

  • Image inference:
python imagePoseEstimation.py
  • Video inference:
python videoPoseEstimation.py
  • Webcam inference:
python webcamPoseEstimation.py

Inference video Example

Mobile Human 3D Pose mation ONNX

References:

You might also like...
Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite.
Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite.

TFLite-HITNET-Stereo-depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite. Stereo depth e

Ibai Gorordo 22 Oct 20, 2022
Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite.
Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite.

TFLite-MobileStereoNet Python scripts for performing stereo depth estimation using the MobileStereoNet model in Tensorflow Lite. Stereo depth estimati

Ibai Gorordo 4 Feb 14, 2022
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX

ONNX msg_chn_wacv20 depth completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20 model in

Ibai Gorordo 19 Oct 22, 2022
This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).
This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).

Github Code of "MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices" Introduction This repo is official PyTorch implementatio

Choi Sang Bum 203 Jan 5, 2023
An executor that loads ONNX models and embeds documents using the ONNX runtime.

ONNXEncoder An executor that loads ONNX models and embeds documents using the ONNX runtime. Usage via Docker image (recommended) from jina import Flow

Jina AI 2 Mar 15, 2022
Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.
Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Example scripts for the detection of lanes using the ultra fast lane detection model in ONNX.

Ibai Gorordo 35 Sep 7, 2022
ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS.

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS. It currently supports four examples for you to quickly experience the power of ONNX Runtime Web.

Microsoft 58 Dec 18, 2022
A repository that shares tuning results of trained models generated by TensorFlow / Keras. Post-training quantization (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization), Quantization-aware training. TensorFlow Lite. OpenVINO. CoreML. TensorFlow.js. TF-TRT. MediaPipe. ONNX. [.tflite,.h5,.pb,saved_model,tfjs,tftrt,mlmodel,.xml/.bin, .onnx]
A repository that shares tuning results of trained models generated by TensorFlow / Keras. Post-training quantization (Weight Quantization, Integer Quantization, Full Integer Quantization, Float16 Quantization), Quantization-aware training. TensorFlow Lite. OpenVINO. CoreML. TensorFlow.js. TF-TRT. MediaPipe. ONNX. [.tflite,.h5,.pb,saved_model,tfjs,tftrt,mlmodel,.xml/.bin, .onnx]

PINTO_model_zoo Please read the contents of the LICENSE file located directly under each folder before using the model. My model conversion scripts ar

Katsuya Hyodo 2.4k Jan 5, 2023
A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want.
A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want.

sne4onnx A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or

Katsuya Hyodo 10 Aug 30, 2022
Comments
  • [Question] What kind of idea do you base the estimation of depth

    [Question] What kind of idea do you base the estimation of depth

    Thank you for sharing. Could you tell me the idea behind the following line? https://github.com/ibaiGorordo/ONNX-Mobile-Human-Pose-3D/blob/main/mobileHumanPose/mobileHumanPose.py#L161

    I understand you need abs_depth. For instance, using the average height will be one of solutions. focal_length(fixed value) : abs_depth(unknown value) = height_in_pixel(known value) : average_height(fixed value) Using the above equation, we can get abs_depth. (It may not work well, though)

    In your code, it looks you use area, and add +500. The values may be determined by experiments, but could you tell me the basic idea of this?

    opened by iwatake2222 2
  • Inference problem

    Inference problem

    Hi,thanks for the great work. I follow your Readme and download the required models including Mobile Human Pose and YOLOv5s, and below is the inference result and which is not as accurate as i expect. The input image size is 512*512, and i have no idea to choose the proper detection model(different resolution),so i just tried it all,which not improved the result.Could you point out what the problem is?

    output

    opened by SienChen1997 2
  • Error downloading model file（mobile_human_pose_working_well_256x256.onnx）？

    Error downloading model file（mobile_human_pose_working_well_256x256.onnx）？

    I can't download through the link（https://github.com/PINTO0309/PINTO_model_zoo/blob/main/135_CoEx/download.sh）.The error screenshot is as follows： 09d4da6e7f4c29ed57d53742fc38d16 ，I can not find the cookie file.Can you provide an absolute download link，thank you.

    opened by shnulailai 2
  • How to improve the quality of keypoints 2D and 3D?

    How to improve the quality of keypoints 2D and 3D?

    I have tested on rtsp live feed to get 2D and 3D poses. but i see so much variation on keypoints extracted due to which even 2D visualization is not looking good. How can we make it better, is there any better model file available. and what is role of Focal length and principal points in computation because i see they are commented in pix2cam function.

    opened by chandelarvind 0
Owner
Ibai Gorordo
Passionate about sensors, technology and their potential to help people.
Ibai Gorordo
GitHub
ONNX-PackNet-SfM: Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Ibai Gorordo 14 Dec 9, 2022
Python scripts form performing stereo depth estimation using the CoEx model in ONNX.

ONNX-CoEx-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the CoEx model in ONNX. Stereo depth estimation on the

Ibai Gorordo 8 Dec 29, 2022
Python scripts form performing stereo depth estimation using the HITNET model in ONNX.

ONNX-HITNET-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in ONNX. Stereo depth estimation on

Ibai Gorordo 30 Nov 8, 2022
Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

ONNX-MobileStereoNet Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX Stereo depth estimation on the cone

Ibai Gorordo 23 Nov 29, 2022
Python scripts for performing lane detection using the LSTR model in ONNX

ONNX LSTR Lane Detection Python scripts for performing lane detection using the Lane Shape Prediction with Transformers (LSTR) model in ONNX. Requirem

Ibai Gorordo 29 Aug 30, 2022
Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONNX.

ONNX-HybridNets-Multitask-Road-Detection Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONN

Ibai Gorordo 45 Jan 1, 2023
Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

ONNX Object Localization Network Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX. Ori

Ibai Gorordo 15 Oct 14, 2022
Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX.

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX. The repository combines a class agnostic object localizer to first detect the objects in the image, and next a ResNet50 model trained on ImageNet is used to label each box.

Ibai Gorordo 24 Nov 14, 2022
Python scripts for performing stereo depth estimation using the HITNET Tensorflow model.

HITNET-Stereo-Depth-estimation Python scripts for performing stereo depth estimation using the HITNET Tensorflow model from Google Research. Stereo de

Ibai Gorordo 76 Jan 2, 2023
Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch .

PyTorch-High-Res-Stereo-Depth-Estimation Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch. Stereo dep

Ibai Gorordo 26 Nov 24, 2022