A simple baseline for 3d human pose estimation in PyTorch.

Overview

3d_pose_baseline_pytorch

A PyTorch implementation of a simple baseline for 3d human pose estimation. You can check the original Tensorflow implementation written by Julieta Martinez et al.. Some codes for data processing are brought from the original version, thanks to the authors.

This is the code for the paper

@inproceedings{martinez_2017_3dbaseline,
  title={A simple yet effective baseline for 3d human pose estimation},
  author={Martinez, Julieta and Hossain, Rayat and Romero, Javier and Little, James J.},
  booktitle={ICCV},
  year={2017}
}

WIP

  • Training code
  • Testing code

Datasets

  • Human3.6M
  • HumanEva

Dependencies

Installation

  1. First, clone this repository:
    git clone --recursive https://github.com/weigq/3d_pose_baseline_pytorch.git
    
  2. Download the pre-processed Human3.6M dataset in 3d joints:
    unzip human36m.zip
    rm h36m.zip
    

Usage

Data preprocess

Train

  1. Train on Human3.6M groundtruth 2d joints:

    # optional arguments, you can access more details in opt.py
    main.py [-h] [--data_dir DATA_DIR] [--exp EXP] [--ckpt CKPT]
               [--load LOAD] [--test] [--resume]
               [--action {all,All}]
               [--max_norm] [--linear_size LINEAR_SIZE]
               [--num_stage NUM_STAGE] [--use_hg] [--lr LR]
               [--lr_decay LR_DECAY] [--lr_gamma LR_GAMMA] [--epochs EPOCHS]
               [--dropout DROPOUT] [--train_batch TRAIN_BATCH]
               [--test_batch TEST_BATCH] [--job JOB] [--no_max] [--max]
               [--procrustes]
    

    train the model:

    python main.py --exp example
    

    You will get the training and testing loss curves like:

    log

  2. Train on Human3.6M 2d joints detected by stacked hourglass:

Test

  1. You can download the pretrained model on ground-truth 2d pose for a quick demo.

    python main.py --load $PATH_TO_gt_ckpt_best.pth.tar --test
    

    and you will get the results:

    direct. discuss. eat. greet. phone photo pose purch. sit sitd. somke wait walkd. walk walkT avg
    original version 37.7 44.4 40.3 42.1 48.2 54.9 44.4 42.1 54.6 58.0 45.1 46.4 47.6 36.4 40.4 45.5
    pytorch version 35.7 42.3 39.4 40.7 44.5 53.3 42.8 40.1 52.5 53.9 42.8 43.1 44.1 33.4 36.3 -

License

MIT

Comments
  • About the result of the model

    About the result of the model

    Hi, Thanks for your codes. I wonder why the results of "original version" are different from the paper and the pytorch version is a little better than the paper?

    Thanks a lot!

    opened by dmortem 14
  • Stacked Hourglass Detections

    Stacked Hourglass Detections

    Hi @weigq,

    Thanks for the great implementation. I'm trying to train on SHG detections but the .zip file doesn't contain the test_2d_ft.pth.tar file.

    Could you provide detection files or processing script for that?

    opened by mkocabas 10
  • Problem on Visualizing

    Problem on Visualizing

    I follow with https://github.com/weigq/3d_pose_baseline_pytorch/blob/a03094bb479770abc43dc5b87ad394b85bb6dcab/main.py#L226 to unnorm the output of network, and get tensor which has (51, ) shape. I guess it should be the coordinates like x0,y0,z0,x1,y1,z1.......x16,y16,z16 but when I draw all 17 point in 3d plot, it don't like a human skeleton shape,

    I want to know how to visualize the output from the (51, ) array, and the index of these 51 nums.

    opened by ghost 9
  • Pre-processed data

    Pre-processed data

    Hi there,

    Your code looks great. I was just wondering what is the main difference between your pre-processed dataset and h36m.zip from the original tensorflow repo.

    Thanks

    Useful issue 
    opened by macaodha 9
  • Unnormalize and Normalize 2D Skeleton

    Unnormalize and Normalize 2D Skeleton

    Hi, since the dataset provided here is normalized version, could anyone give me idea how to unnormalize 2D Skeleton? I already see this function and successfully unnormalize the 3D Skeleton, but how we use this code to unnormalize 2D Skeleton? I will really glad if someone could help. Also is there a function to do normalize? Thank you.

    def unNormalizeData(normalized_data, data_mean, data_std, dimensions_to_use):

    opened by alexivaner 6
  • Hourglass detection files

    Hourglass detection files

    Hello,

    The SH detections do not seem to be available anymore. Also in the main repo (https://github.com/una-dinosauria/3d-pose-baseline) there is no code available for hourglass detections. Does anyone happen to have the detection files, i.e. 'train_2d_ft.pth.tar' and 'test_2d_ft.pth.tar'? Or the code for the stacked hourglass model that was used to produce the results?

    At the moment I am only able to run my experiments with the GT. It would be great if someone could help me with the SH detections :)

    Cheers, Duncan

    opened by DuncanZauss 5
  • About visualization

    About visualization

    Thanks for the great repo! Your code is very elegant. Just a quick question. I am trying to plot the given 2d and 3d processed skeletons. However, I could not find the connection between joints in your repo. I am wondering if you could either provide the limb connections or the code for visualizing?

    Thanks a lot!

    opened by zc-alexfan 3
  • How?

    How?

    dear @weigq, What is that?

    optional arguments, you can access more details in opt.py

    main.py [-h] [--data_dir DATA_DIR] [--exp EXP] [--ckpt CKPT] [--load LOAD] [--test] [--resume] [--action {all,All}] [--max_norm] [--linear_size LINEAR_SIZE] [--num_stage NUM_STAGE] [--use_hg] [--lr LR] [--lr_decay LR_DECAY] [--lr_gamma LR_GAMMA] [--epochs EPOCHS] [--dropout DROPOUT] [--train_batch TRAIN_BATCH] [--test_batch TEST_BATCH] [--job JOB] [--no_max] [--max] [--procrustes]

    opened by NguyenDangBinh 3
  • Bug in Error Calculation

    Bug in Error Calculation

    Hi, Wanted to point out a bug in your error calculation. The errors for all actions are simply averaged, without regard to the number of data points in each action. It should be a weighted mean. This yields slightly higher numbers.

    opened by ssfootball04 3
  • Error computation

    Error computation

    Hi there,

    Thanks again for sharing your code, its very clear.

    I have a quick question about the error computation. You select the valid dimensions here which includes dimensions for the hip (which is not predicted) for. As I understand it, in the squared error computation three of the dimensions will always have a very low error e.g. they are predicted zero and the gt is zeros.

    Is this the same way as the original TensorFlow code?

    Thanks

    opened by macaodha 2
  • Joints Order

    Joints Order

    @weigq Hello! I've run train_2d_ft.pth.tar and test_2d_ft.pth.tar on other models, whose performance is very poor. So I want to know if the train_2d_ft.pth.tar contains the same kind of joints and the same order of joints as it did on train_2d.pth.tar and train_3d.pth.tar ? Thanks!

    opened by guoyuhang1996 1
  • Questions on  visualizing and data pre-processing.

    Questions on visualizing and data pre-processing.

    Hi! I'm confused when visualizing the poses. For example, I get a numpy.ndarray from test_2d.pth.tar, which has the shape (32,) and represent a 2d pose. I guess it arrays as x1,y1,x2,y2, ..., so I add 0,0 at the front of pose, and try to connect them following the sequence of the original version , but it dosen't look like a human. The same result on another arrayx1,x2,...,y1,y2, ..., and 3d pose as well. I want to know whether I misunderstanding some thing in the process above. And what's the shape of normalized bbox from original picture? How to change the coordinates from pixel to the form we see now in the files? Thanks!!!

    opened by gaoyh18 1
  • data is missing in the git repo

    data is missing in the git repo

    Could you please share the data/stat_3d.pth.tar ?

    [jalal@goku 3d_pose_baseline_pytorch]$ python main.py --exp example
    
    ==================Options=================
    {   'action': 'All',
        'ckpt': 'checkpoint/example',
        'data_dir': 'data/',
        'dropout': 0.5,
        'epochs': 200,
        'exp': 'example',
        'is_train': True,
        'job': 8,
        'linear_size': 1024,
        'load': '',
        'lr': 0.001,
        'lr_decay': 100000,
        'lr_gamma': 0.96,
        'max_norm': True,
        'num_stage': 2,
        'procrustes': False,
        'resume': False,
        'test': False,
        'test_batch': 64,
        'train_batch': 64,
        'use_hg': False}
    ==========================================
    
    >>> creating model
    /scratch3/3d_pose/humanpose/3d_pose_baseline_pytorch/src/model.py:11: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
      nn.init.kaiming_normal(m.weight)
    >>> total params: 4.29M
    /scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:52: UserWarning: size_average and reduce args will be deprecated, please use reduction='elementwise_mean' instead.
      warnings.warn(warning.format(ret))
    >>> actions to use (total: 15):
    [   'Directions',
        'Discussion',
        'Eating',
        'Greeting',
        'Phoning',
        'Photo',
        'Posing',
        'Purchases',
        'Sitting',
        'SittingDown',
        'Smoking',
        'Waiting',
        'WalkDog',
        'Walking',
        'WalkTogether']
    >>>
    >>> loading data
    Traceback (most recent call last):
      File "main.py", line 276, in <module>
        main(option)
      File "main.py", line 75, in main
        stat_3d = torch.load(os.path.join(opt.data_dir, 'stat_3d.pth.tar'))
      File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
        f = open(f, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: 'data/stat_3d.pth.tar'
    
    opened by monajalal 6
  • How to get the train data and test data?

    How to get the train data and test data?

    how to get the data train_2d_ft.pth.tar, test_2d_ft.pth.tar, train_2d.pth.tar,test_2d.pth.tar and train_3d.pth.tar ? Thank you so much If you can help me !

    opened by wongkwan 2
Owner
weigq
weigq
This is an official implementation for "DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation"

DeciWatch: A Simple Baseline for 10× Efficient 2D and 3D Pose Estimation This repo is the official implementation of "DeciWatch: A Simple Baseline for

null 117 Dec 24, 2022
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

Aymen Mir 66 Dec 21, 2022
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

Jia Li 256 Dec 24, 2022
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022
PyTorch implementation for 3D human pose estimation

Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach This repository is the PyTorch implementation for the network presented in:

Xingyi Zhou 579 Dec 22, 2022
A PyTorch toolkit for 2D Human Pose Estimation.

PyTorch-Pose PyTorch-Pose is a PyTorch implementation of the general pipeline for 2D single human pose estimation. The aim is to provide the interface

Wei Yang 1.1k Dec 30, 2022
This repository is the offical Pytorch implementation of ContextPose: Context Modeling in 3D Human Pose Estimation: A Unified Perspective (CVPR 2021).

Context Modeling in 3D Human Pose Estimation: A Unified Perspective (CVPR 2021) Introduction This repository is the offical Pytorch implementation of

null 37 Nov 21, 2022
This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).

Github Code of "MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices" Introduction This repo is official PyTorch implementatio

Choi Sang Bum 203 Jan 5, 2023
Image-retrieval-baseline - MUGE Multimodal Retrieval Baseline

MUGE Multimodal Retrieval Baseline This repo is implemented based on the open_cl

null 47 Dec 16, 2022
Image-generation-baseline - MUGE Text To Image Generation Baseline

MUGE Text To Image Generation Baseline Requirements and Installation More detail

null 23 Oct 17, 2022
Jingju baseline - A baseline model of our project of Beijing opera script generation

Jingju Baseline It is a baseline of our project about Beijing opera script gener

midon 1 Jan 14, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

shangbuhuan 52 Nov 25, 2022
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 71 Jan 5, 2023
Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021) Introduction This is the official code of Deep Dual Consecutive Network for Human P

null 295 Dec 29, 2022
Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

null 108 Dec 1, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022