Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows

Related tags

Deep Learning Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows
Overview

Probabilistic-Monocular-3D-Human-Pose-Estimation-with-Normalizing-Flows

This is the official implementation of the ICCV 2021 Paper "Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows" by Tom Wehrbein, Marco Rudolph, Bodo Rosenhahn and Bastian Wandt.

Installation instructions

We recommend creating a clean conda environment. You can do this as follows:

conda env create -f environment.yml

After the installation is complete, you can activate the conda environment by running:

conda activate ProHPE

Finally, FrEIA needs to be installed:

pip install git+https://github.com/VLL-HD/FrEIA.git@ad2f5a261a2fc0002fb4c9adeff7a62b0e9dd4e1

Data

Download the used Human3.6M detections and Gaussian fits from Google Drive. Afterwards, extract the zip file to the data/ directory.

Due to licensing, it is not possible for us to provide any data from Human3.6M. Therefore, the 3D pose data needs to be downloaded from the official dataset website (account required). For each subject, download the file 'D3_Positions_mono' and extract it to the data/ directory. Afterwards, run

python create_data.py

to automatically merge the detections and Gaussian fits with the 3D ground truth data.

For information about the dataset structure, see data/data_info.txt.

Run evaluation code

Evaluate the pretrained model on the whole testset of Human3.6M:

python eval_action_wise.py --exp original_model

Evaluate on the hard subset of Human3.6M containing highly ambiguous examples:

python eval_hard_subset.py --exp original_model

Evaluation results are saved to the results/original_model/ directory.

Run training code

Training can be started with:

python train.py --exp experiment_name

Citation

Please cite the paper in your publications if it helps your research:

@inproceedings {WehRud2021,
  author = {Tom Wehrbein and Marco Rudolph and Bodo Rosenhahn and Bastian Wandt},
  title = {Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows},
  booktitle = {International Conference on Computer Vision (ICCV)},
  year = {2021},
  month = oct
}

Link to the paper:

You might also like...
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

Jia Li 256 Dec 24, 2022
This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

shangbuhuan 52 Nov 25, 2022
The project is an official implementation of our CVPR2019 paper
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Human head pose estimation using Keras over TensorFlow.
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 71 Jan 5, 2023
Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)
Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Deep Dual Consecutive Network for Human Pose Estimation （CVPR2021） Introduction This is the official code of Deep Dual Consecutive Network for Human P

null 295 Dec 29, 2022
Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

null 108 Dec 1, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation Official PyTroch implementation of HPRNet. HPRNet: Hierarchical Point Regre

Nermin Samet 53 Dec 4, 2022
Comments
  • Provided pretrained model is not loading

    Provided pretrained model is not loading

    I am trying to run your evaluation code on the pre-trained model.

    I am running the python eval_action_wise.py --exp original_model command as suggested in your README.

    However, I am getting the following error.

    python eval_action_wise.py --exp original_model

Program is running on:  cpu
EVALUATING EXPERIMENT:  original_model 

Traceback (most recent call last):
  File "/Users/paulpierzchlewicz/PycharmProjects/ProHPE/eval_action_wise.py", line 21, in <module>
    inn.load(c.load_model_name, c.device)
  File "/Users/paulpierzchlewicz/PycharmProjects/ProHPE/models/model.py", line 63, in load
    self.load_state_dict(network_state_dict)
  File "/Users/paulpierzchlewicz/PycharmProjects/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(

RuntimeError: Error(s) in loading state_dict for poseINN:
        Missing key(s) in state_dict: "inn.module_list.0.subnet1.0.weight", "inn.module_list.0.subnet1.0.bias", "inn.module_list.0.subnet1.2.weight", "inn.module_list.0.subnet1.2.bias", "inn.module_list.0.subnet2.0.weight", "inn.module_list.0.subnet2.0.bias", "inn.module_list.0.subnet2.2.weight", "inn.module_list.0.subnet2.2.bias", "inn.module_list.1.perm", "inn.module_list.1.perm_inv", "inn.module_list.2.subnet1.0.weight", "inn.module_list.2.subnet1.0.bias", "inn.module_list.2.subnet1.2.weight", "inn.module_list.2.subnet1.2.bias", "inn.module_list.2.subnet2.0.weight", "inn.module_list.2.subnet2.0.bias", "inn.module_list.2.subnet2.2.weight", "inn.module_list.2.subnet2.2.bias", "inn.module_list.3.perm", "inn.module_list.3.perm_inv", "inn.module_list.4.subnet1.0.weight", "inn.module_list.4.subnet1.0.bias", "inn.module_list.4.subnet1.2.weight", "inn.module_list.4.subnet1.2.bias", "inn.module_list.4.subnet2.0.weight", "inn.module_list.4.subnet2.0.bias", "inn.module_list.4.subnet2.2.weight", "inn.module_list.4.subnet2.2.bias", "inn.module_list.5.perm", "inn.module_list.5.perm_inv", "inn.module_list.6.subnet1.0.weight", "inn.module_list.6.subnet1.0.bias", "inn.module_list.6.subnet1.2.weight", "inn.module_list.6.subnet1.2.bias", "inn.module_list.6.subnet2.0.weight", "inn.module_list.6.subnet2.0.bias", "inn.module_list.6.subnet2.2.weight", "inn.module_list.6.subnet2.2.bias", "inn.module_list.7.perm", "inn.module_list.7.perm_inv", "inn.module_list.8.subnet1.0.weight", "inn.module_list.8.subnet1.0.bias", "inn.module_list.8.subnet1.2.weight", "inn.module_list.8.subnet1.2.bias", "inn.module_list.8.subnet2.0.weight", "inn.module_list.8.subnet2.0.bias", "inn.module_list.8.subnet2.2.weight", "inn.module_list.8.subnet2.2.bias", "inn.module_list.9.perm", "inn.module_list.9.perm_inv", "inn.module_list.10.subnet1.0.weight", "inn.module_list.10.subnet1.0.bias", "inn.module_list.10.subnet1.2.weight", "inn.module_list.10.subnet1.2.bias", "inn.module_list.10.subnet2.0.weight", "inn.module_list.10.subnet2.0.bias", "inn.module_list.10.subnet2.2.weight", "inn.module_list.10.subnet2.2.bias", "inn.module_list.11.perm", "inn.module_list.11.perm_inv", "inn.module_list.12.subnet1.0.weight", "inn.module_list.12.subnet1.0.bias", "inn.module_list.12.subnet1.2.weight", "inn.module_list.12.subnet1.2.bias", "inn.module_list.12.subnet2.0.weight", "inn.module_list.12.subnet2.0.bias", "inn.module_list.12.subnet2.2.weight", "inn.module_list.12.subnet2.2.bias", "inn.module_list.13.perm", "inn.module_list.13.perm_inv", "inn.module_list.14.subnet1.0.weight", "inn.module_list.14.subnet1.0.bias", "inn.module_list.14.subnet1.2.weight", "inn.module_list.14.subnet1.2.bias", "inn.module_list.14.subnet2.0.weight", "inn.module_list.14.subnet2.0.bias", "inn.module_list.14.subnet2.2.weight", "inn.module_list.14.subnet2.2.bias", "inn.module_list.15.perm", "inn.module_list.15.perm_inv". 
        Unexpected key(s) in state_dict: "inn.module_list.1.s1.0.weight", "inn.module_list.1.s1.0.bias", "inn.module_list.1.s1.2.weight", "inn.module_list.1.s1.2.bias", "inn.module_list.1.s2.0.weight", "inn.module_list.1.s2.0.bias", "inn.module_list.1.s2.2.weight", "inn.module_list.1.s2.2.bias", "inn.module_list.3.s1.0.weight", "inn.module_list.3.s1.0.bias", "inn.module_list.3.s1.2.weight", "inn.module_list.3.s1.2.bias", "inn.module_list.3.s2.0.weight", "inn.module_list.3.s2.0.bias", "inn.module_list.3.s2.2.weight", "inn.module_list.3.s2.2.bias", "inn.module_list.5.s1.0.weight", "inn.module_list.5.s1.0.bias", "inn.module_list.5.s1.2.weight", "inn.module_list.5.s1.2.bias", "inn.module_list.5.s2.0.weight", "inn.module_list.5.s2.0.bias", "inn.module_list.5.s2.2.weight", "inn.module_list.5.s2.2.bias", "inn.module_list.7.s1.0.weight", "inn.module_list.7.s1.0.bias", "inn.module_list.7.s1.2.weight", "inn.module_list.7.s1.2.bias", "inn.module_list.7.s2.0.weight", "inn.module_list.7.s2.0.bias", "inn.module_list.7.s2.2.weight", "inn.module_list.7.s2.2.bias", "inn.module_list.9.s1.0.weight", "inn.module_list.9.s1.0.bias", "inn.module_list.9.s1.2.weight", "inn.module_list.9.s1.2.bias", "inn.module_list.9.s2.0.weight", "inn.module_list.9.s2.0.bias", "inn.module_list.9.s2.2.weight", "inn.module_list.9.s2.2.bias", "inn.module_list.11.s1.0.weight", "inn.module_list.11.s1.0.bias", "inn.module_list.11.s1.2.weight", "inn.module_list.11.s1.2.bias", "inn.module_list.11.s2.0.weight", "inn.module_list.11.s2.0.bias", "inn.module_list.11.s2.2.weight", "inn.module_list.11.s2.2.bias", "inn.module_list.13.s1.0.weight", "inn.module_list.13.s1.0.bias", "inn.module_list.13.s1.2.weight", "inn.module_list.13.s1.2.bias", "inn.module_list.13.s2.0.weight", "inn.module_list.13.s2.0.bias", "inn.module_list.13.s2.2.weight", "inn.module_list.13.s2.2.bias", "inn.module_list.15.s1.0.weight", "inn.module_list.15.s1.0.bias", "inn.module_list.15.s1.2.weight", "inn.module_list.15.s1.2.bias", "inn.module_list.15.s2.0.weight", "inn.module_list.15.s2.0.bias", "inn.module_list.15.s2.2.weight", "inn.module_list.15.s2.2.bias".

    Loading using self.load_state_dict(network_state_dict, strict=False) in ./models/model.py line 63 seems to fix the problem, but then the evaluation metrics are off by a lot.

    Average:
3D Protocol-I z_0: 1761697.93
3D Protocol-I best hypo: 73987.92
3D Protocol-I median hypo: 2354026.03
3D Protocol-I mean hypo: 4192725.50
3D Protocol-I worst hypo: 49519483.01

3D Protocol-II z_0: 408.56
3D Protocol-II best hypo: 360.55
3D Protocol-II median hypo: 408.01
3D Protocol-II mean hypo: 406.66
3D Protocol-II worst hypo: 429.66

std dev per joint and dim in mm:
joint 0: std_x=0.00, std_y=0.00, std_z=0.00
joint 1: std_x=2713363.25, std_y=4497608.00, std_z=1229723.62
joint 2: std_x=5092118.50, std_y=5026269.00, std_z=3333644.75
joint 3: std_x=3927144.75, std_y=4752659.00, std_z=4638813.50
joint 4: std_x=2713363.25, std_y=4497608.00, std_z=1229723.62
joint 5: std_x=4988073.00, std_y=4202771.00, std_z=1055546.38
joint 6: std_x=4485964.50, std_y=5458925.00, std_z=1890290.62
joint 7: std_x=4712188.50, std_y=5419219.50, std_z=1223268.88
joint 8: std_x=3895103.75, std_y=6552087.00, std_z=1079990.88
joint 9: std_x=4310205.00, std_y=4260656.50, std_z=7185773.00
joint 10: std_x=7090483.50, std_y=5303002.50, std_z=1693970.62
joint 11: std_x=4958182.00, std_y=4558726.00, std_z=1272373.12
joint 12: std_x=4196097.00, std_y=4670686.50, std_z=1536391.38
joint 13: std_x=3967276.75, std_y=4828261.50, std_z=1302280.12
joint 14: std_x=8149503.50, std_y=4338247.50, std_z=1942508.12
joint 15: std_x=11636298.00, std_y=6795918.00, std_z=1937581.38
joint 16: std_x=4513685.00, std_y=917258.38, std_z=1015419.19

    Would you maybe know what the issue might be?

    opened by PPierzc 2
  • 2D and 3D input normalization with mean and std.

    2D and 3D input normalization with mean and std.

    I noticed in data/data_h36m.py, you normalized the input 2D and 3D with mean and std with specific axes. Can I ask why this effect is too much to the results when I do not use this normalized function? Furthermore, why you would use invert y,z axes by multiplying them with -1?

    opened by huyphan168 1
  • How to visualize the result of Fig3

    How to visualize the result of Fig3

    Hi, author~ Excellent work, and thanks for releasing your code.

    I want to visualize the result, just like what you do in Fig3.
    I have tried your "viz/plot_p{2d/3d}.py" code, but nothing appears at all. Can you explain visualize step for your code?

    Thanks!

    opened by wenwen12321 0
Owner
null
GitHub
Stochastic Normalizing Flows

Stochastic Normalizing Flows We introduce stochasticity in Boltzmann-generating flows. Normalizing flows are exact-probability generative models that

AI4Science group, FU Berlin (Frank Noé and co-workers) 50 Dec 16, 2022
Official PyTorch code for WACV 2022 paper "CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows"

CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows WACV 2022 preprint:https://arxiv.org/abs/2107.1

Denis 156 Dec 28, 2022
TensorFlow implementation of "Variational Inference with Normalizing Flows"

[TensorFlow 2] Variational Inference with Normalizing Flows TensorFlow implementation of "Variational Inference with Normalizing Flows" [1] Concept Co

YeongHyeon Park 7 Jun 8, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

Aymen Mir 66 Dec 21, 2022
Official project website for the CVPR 2021 paper "Exploring intermediate representation for monocular vehicle pose estimation"

EgoNet Official project website for the CVPR 2021 paper "Exploring intermediate representation for monocular vehicle pose estimation". This repo inclu

Shichao Li 138 Dec 9, 2022
GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation. (CVPR 2021)

GDR-Net This repo provides the PyTorch implementation of the work: Gu Wang, Fabian Manhardt, Federico Tombari, Xiangyang Ji. GDR-Net: Geometry-Guided

null 169 Jan 7, 2023
Monocular 3D pose estimation. OpenVINO. CPU inference or iGPU (OpenCL) inference.

human-pose-estimation-3d-python-cpp RealSenseD435 (RGB) 480x640 + CPU Corei9 45 FPS (Depth is not used) 1. Run 1-1. RealSenseD435 (RGB) 480x640 + CPU

Katsuya Hyodo 8 Oct 3, 2022
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Pranav 39 Nov 21, 2022
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022