Source code release of the paper: Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation.

Related tags

Deep Learning GNet-pose
Overview

GNet-pose

Project Page: http://guanghan.info/projects/guided-fractal/

UPDATE 9/27/2018:

Prototxts and model that achieved 93.9Pck on LSP dataset. http://guanghan.info/download/Data/GNet_update.zip

When I was replying e-mails, it occurred to me that the models that I had uploaded was around May/June 2017 (performance in old arxiv version), and in August 2017 the performance was improved to 93.9 on LSP with a newer caffe version which fixed the downsampling and/or upsampling deprecation problem (Yeah, it "magically" improved the performance). The best model was 94.0071 on LSP dataset, but it was not uploaded nor published on the benchmark.

Overview

Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation.

Source code release of the paper for reproduction of experimental results, and to aid researchers in future research.

Prerequisites

Getting Started

1. Download Data and Pre-trained Models

  • Datasets (MPII [1], LSP [2])

    bash ./get_dataset.sh

  • Models

    bash ./get_models.sh

  • Predictions (optional)

    bash ./get_preds.sh

2. Testing

  • Generate cropped patches from the dataset for testing:

    cd testing/
matlab gen_cropped_LSP_test_images.m
matlab gen_cropped_MPII_test_images.m
cd -

    This will generate images with 368-by-368 resolution.

  • Reproduce the results with the pre-trained model:

    cd testing/
python .test.py
cd -

    You can choose different dataset to test on, with different models. You can also choose different settings in test.py, e.g., with or without flipping, scaling, cross-heatmap regression, etc.

3. Training

  • Generate Annotations

    cd training/Annotations/
matlab MPI.m LEEDS.m
cd -

    This will generate annotations in json files.

  • Generate LMDB

    python ./training/Data/genLMDB.py

    This will load images from dataset and annotations from json files, and generate lmdb files for caffe training.

  • Generate Prototxt files (optional)

    python ./training/GNet/scripts/gen_GNet.py
python ./training/GNet/scripts/gen_fractal.py
python ./training/GNet/scripts/gen_hourglass.py

  • Training:

     bash ./training/train.sh

4. Performance Evaluation

cd testing/eval_LSP/; matlab test_evaluation_lsp.m; cd../

cd testing/eval_MPII/; matlab test_evaluation_mpii_test.m

5. Results

More Qualitative results can be found in the project page. Quantitative results please refer to the arxiv paper.

License

GNet-pose is released under the Apache License Version 2.0 (refer to the LICENSE file for details).

Citation

If you use the code and models, please cite the following paper: TMM 2017.

@article{ning2017knowledge, 
 author={G. Ning and Z. Zhang and Z. He}, 
     journal={IEEE Transactions on Multimedia}, 
     title={Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation}, 
     year={2017}, 
     doi={10.1109/TMM.2017.2762010}, 
     ISSN={1520-9210}, }

Reference

[1] Andriluka M, Pishchulin L, Gehler P, et al. "2d human pose estimation: New benchmark and state of the art analysis." CVPR (2014).

[2] Sam Johnson and Mark Everingham. "Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation." BMVC (2010).

Comments
  • Datset-Download not available any more?

    Datset-Download not available any more?

    I'm trying to download the dataset with get_dataset.sh

    But the link http://www.comp.leeds.ac.uk/mat4saj/lsp_dataset.zip does not get a zip-file but a webpage instead. Somehow this is redirected to: https://engineering.leeds.ac.uk/info/20132/school_of_computing

    Same with: http://www.comp.leeds.ac.uk/mat4saj/lspet_dataset.zip

    How can I download the dataset?

    opened by stvogel 1
  • support for multiple person?

    support for multiple person?

    From the project page, the demo seems support multiple person pose detection , while in the paper, the benchmark images are for single person. Could you please clarify that whether Gnet-pose support multiple person pose detection or not ?

    opened by ouceduxzk 1
  • result images not found!

    result images not found!

    After clearing some errors, I successfully runned the test.py in google colab and finally, it shows the message testing scales: [1, 0.75] output_image_folder_path: dataset_lsp/results_cropped/

    But no images were found in the path. I have attached my input images in the path dataset_lsp/images_cropped/

    Are there any changes I have to do in input images or in its path?

    opened by HARIHARAN1103 0
  • lack of some packbags

    lack of some packbags

    load('eval_MPII/Tompson Test Set', 'annolist'); load('eval_MPII/Tompson Test Set', 'keypointsAll'); load('eval_MPII/Tompson Test Set', 'RELEASE_img_index'); load('eval_MPII/Tompson Test Set', 'RELEASE_person_index'); Where i find it ??

    opened by Easyfeng222 0
  • The loss cannot converge

    The loss cannot converge

    Hi @Guanghan , Thanks for your great project. Can you show to me your loss curve of your final model? Below is my result when training GNet-pose with base_lr: 0.0001, batch_size: 4, after 3 days, trained 76 epochs. 313 It still cannot converge. If you trained with other parameter in prototxt file, please tell me. I'm waiting for your feedback.

    opened by vuonglequoc 1
  • modified caffe

    modified caffe

    hi! your work seems awesome, but it is really difficult to install the modified caffe. is there any chance that you'll add a step by step explanation? or at-least some background as to what is needed before installing it?

    opened by levavtrack 0
  • About testing

    About testing

    Hi, @Guanghan ,Thanks your great repo , when I run test.py , the show of result looks like bad , I guess it may have relationship with the image resolution , so ,how size the imput image be set?

    opened by feitiandemiaomi 0
Owner
Guanghan Ning
Guanghan Ning
GitHub
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations at CVPR'21. According to some product reasons, we are not planning to release the training/testing codes and models. However, we will release the dataset and the scripts to prepare the dataset.

TransFill-Reference-Inpainting This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transf

Yuqian Zhou 80 Dec 8, 2022
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

Aymen Mir 66 Dec 21, 2022
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
PyTorch implemention of ICCV'21 paper SGPA: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation

SGPA: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation This is the PyTorch implemention of ICCV'21 paper SGPA: Structure

Chen Kai 24 Dec 5, 2022
Code for ICCV 2021 paper "HuMoR: 3D Human Motion Model for Robust Pose Estimation"

Code for ICCV 2021 paper "HuMoR: 3D Human Motion Model for Robust Pose Estimation"

Davis Rempe 367 Dec 24, 2022
This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

shangbuhuan 52 Nov 25, 2022
Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Deep Dual Consecutive Network for Human Pose Estimation （CVPR2021） Introduction This is the official code of Deep Dual Consecutive Network for Human P

null 295 Dec 29, 2022
Deep High-Resolution Representation Learning for Human Pose Estimation

Deep High-Resolution Representation Learning for Human Pose Estimation (accepted to CVPR2019) News If you are interested in internship or research pos

HRNet 167 Dec 27, 2022
Code for the paper "JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design"

JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design This repository contains code for the paper: JA

Aspuru-Guzik group repo 55 Nov 29, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 363 Dec 28, 2022
GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation. (CVPR 2021)

GDR-Net This repo provides the PyTorch implementation of the work: Gu Wang, Fabian Manhardt, Federico Tombari, Xiangyang Ji. GDR-Net: Geometry-Guided

null 169 Jan 7, 2023
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022
Code for "Human Pose Regression with Residual Log-likelihood Estimation", ICCV 2021 Oral

Human Pose Regression with Residual Log-likelihood Estimation [Paper] [arXiv] [Project Page] Human Pose Regression with Residual Log-likelihood Estima

JeffLi 347 Dec 24, 2022
Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

XingBowen 4 May 20, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

Jia Li 256 Dec 24, 2022
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 71 Jan 5, 2023