HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Related tags

Deep Learning HPRNet
Overview

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Official PyTroch implementation of HPRNet.

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation,
Nermin Samet, Emre Akbas,
Under review. (arXiv pre-print)

Highlights

  • HPRNet is a bottom-up, one-stage and hierarchical keypoint regression method for whole-body pose estimation.
  • HPRNet has the best performance among bottom-up methods for all the whole-body parts.
  • HPRNet achieves SOTA performance for the face (76.0 AP) and hand (51.2 AP) keypoint estimation.
  • Unlike two-stage methods, HPRNet predicts whole-body pose in a constant time independent of the number of people in an image.

COCO-WholeBody Keypoint Estimation Results

Model Body AP Foot AP Face AP Hand AP Whole-body AP Download
HPRNet (DLA) 55.2 / 57.1 49.1 / 50.7 74.6 / 75.4 47.0 / 48.4 31.5 / 32.7 model
HPRNet (Hourglass) 59.4 / 61.1 53.0 / 53.9 75.4 / 76.0 50.4 / 51.2 34.8 / 34.9 model
  • Results are presented without and with test time flip augmentation respectively.
  • All models are trained on COCO-WholeBody train2017 and evaluated on val2017.
  • The models can be downloaded directly from Google drive.

Installation

  1. [Optional but recommended] create a new conda environment.

    conda create --name HPRNet python=3.7
    

    And activate the environment.

    conda activate HPRNet
    
  2. Clone the repo:

    HPRNet_ROOT=/path/to/clone/HPRNet
    git clone https://github.com/nerminsamet/HPRNet $HPRNet_ROOT
    
  3. Install PyTorch 1.4.0:

    conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
    
  4. Install the requirements:

    pip install -r requirements.txt
    
  5. Compile DCNv2 (Deformable Convolutional Networks):

    cd $HPRNet_ROOT/src/lib/models/networks/DCNv2
    ./make.sh
    

Dataset preparation

  • Download the images (2017 Train, 2017 Val) from coco website.

  • Download train and val annotation files.

    ${COCO_PATH}
    |-- annotations
        |-- coco_wholebody_train_v1.0.json
        |-- coco_wholebody_val_v1.0.json
    |-- images
        |-- train2017
        |-- val2017 
    

Evaluation and Training

  • You could find all the evaluation and training scripts in the experiments folder.
  • For evaluation, please download the pretrained models you want to evaluate and put them in HPRNet_ROOT/models/.
  • In the case that you don't have 4 GPUs, you can follow the linear learning rate rule to adjust the learning rate.
  • If the training is terminated before finishing, you can use the same command with --resume to resume training.

Acknowledgement

The numerical calculations reported in this paper were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).

License

HPRNet is released under the MIT License (refer to the LICENSE file for details).

Citation

If you find HPRNet useful for your research, please cite our paper as follows:

N. Samet, E. Akbas, "HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation", arXiv, 2021.

BibTeX entry:

@misc{hprnet,
      title={HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation}, 
      author={Nermin Samet and Emre Akbas},
      year={2021}, 
}
You might also like...
WormMovementSimulation - 3D Simulation of Worm Body Movement with Neurons attached to its body
WormMovementSimulation - 3D Simulation of Worm Body Movement with Neurons attached to its body

Generate 3D Locomotion Data This module is intended to create 2D video trajector

Code for
Code for "CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds" @ICRA2021

CloudAAE This is an tensorflow implementation of "CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds" Files log:

Code for
Code for "3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop"

PyMAF This repository contains the code for the following paper: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop Hongwe

GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation. (CVPR 2021)
GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation. (CVPR 2021)

GDR-Net This repo provides the PyTorch implementation of the work: Gu Wang, Fabian Manhardt, Federico Tombari, Xiangyang Ji. GDR-Net: Geometry-Guided

SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

Code for
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Comments
  • Inference visualize

    Inference visualize

    Hi, I have some questions.

    I used detector to visualize, and i got some coord.

    It has {results, tot, load, pre, net, dec, post, merge}, and results has list, its length is 100.

    Please let me know what meaning is the return value of detector.

    If you have time, please give me some hint to make visualization.

    Thank you for your implementations.

    Best Regards.

    opened by apfjdld 3
  • demo

    demo

    greateful work,have you done or will to run completely the demo inference?why the points num is 137, debugger.add_coco_face_lm(bbox[12:149], img_id='landmark', text=ind)?

    opened by wolfworld6 0
Owner
Nermin Samet
PhD candidate
Nermin Samet
Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Ibai Gorordo 99 Dec 31, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
Code for "Human Pose Regression with Residual Log-likelihood Estimation", ICCV 2021 Oral

Human Pose Regression with Residual Log-likelihood Estimation [Paper] [arXiv] [Project Page] Human Pose Regression with Residual Log-likelihood Estima

JeffLi 347 Dec 24, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
Face and Pose detector that emits MQTT events when a face or human body is detected and not detected.

Face Detect MQTT Face or Pose detector that emits MQTT events when a face or human body is detected and not detected. I built this as an alternative t

Jacob Morris 38 Oct 21, 2022
Ultra-lightweight human body posture key point CNN model. ModelSize:2.3MB HUAWEI P40 NCNN benchmark: 6ms/img,

Ultralight-SimplePose Support NCNN mobile terminal deployment Based on MXNET(>=1.5.1) GLUON(>=0.7.0) framework Top-down strategy: The input image is t

null 223 Dec 27, 2022
Demo for Real-time RGBD-based Extended Body Pose Estimation paper

Real-time RGBD-based Extended Body Pose Estimation This repository is a real-time demo for our paper that was published at WACV 2021 conference The ou

Renat Bashirov 118 Dec 26, 2022
pytorch implementation of openpose including Hand and Body Pose Estimation.

pytorch-openpose pytorch implementation of openpose including Body and Hand Pose Estimation, and the pytorch model is directly converted from openpose

Hzzone 1.4k Jan 7, 2023
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image [Project Page] [Paper] [Supp. Mat.] Table of Contents License Description Fittin

Vassilis Choutas 1.3k Jan 7, 2023
Full body anonymization - Realistic Full-Body Anonymization with Surface-Guided GANs

Realistic Full-Body Anonymization with Surface-Guided GANs This is the official

Håkon Hukkelås 30 Nov 18, 2022