The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

Overview

Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

This is the authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

Run Inference for demo (with openpose)

  1. Download openpose pretrained model
  2. Run Inference
    • python bin/demo.py sample/image.png --lift_model sample/gen_epoch_500.npz --model2d pose_iter_440000.caffemodel --proto2d openpose_pose_coco.prototxt
    • Need OpenCV >= 3.4
      • < 3.3 results extreamly wrong estimation

Dependencies(Recommended versions)

Training

Human3.6M dataset

  • Unsupervised learning of 3D points from ground truth 2D points

    python bin/train.py --gpu 0 --mode unsupervised --dataset h36m --use_heuristic_loss
    
  • Unsupervised learning of 3D points from detected 2D points by Stacked Hourglass

    TBA

  • Supervised learning of 3D points from ground truth 2D points

    python bin/train.py --gpu 0 --mode supervised --activate_func relu --use_bn
    

MPII dataset

TBA

MPI-INF-3DHP dataset

TBA

Evaluation

python bin/eval.py results/hoge/gen_epoch_*.npz
Comments
  • Question: _pickle.UnpicklingError: invalid load key, 'v'

    Question: _pickle.UnpicklingError: invalid load key, 'v'

    I tried to run the project on Ubuntu 16.04, but I met an error. The environment is clean installed one. Could you give me some advices on what I should do with this error?

    ----- logs ------ cvl@cvl-All-Series:~/workspace/3dpose_gan$ python bin/demo.py your/image.png --lift_model sample/gen_epoch_500.npz --model pose_iter_440000.caffemodel --proto2d pose_deploy_linevec.prototxt MODEL: generator, N_OUT: 17, N_UNIT: 1024 Traceback (most recent call last): File "/home/cvl/.pyenv/versions/3.6.1/lib/python3.6/site-packages/numpy/lib/npyio.py", line 428, in load return pickle.load(fid, **pickle_kwargs) _pickle.UnpicklingError: invalid load key, 'v'.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "bin/demo.py", line 201, in main(args) File "bin/demo.py", line 157, in main chainer.serializers.load_npz(args.lift_model, model) File "/home/cvl/.pyenv/versions/3.6.1/lib/python3.6/site-packages/chainer/serializers/npz.py", line 151, in load_npz with numpy.load(file) as f: File "/home/cvl/.pyenv/versions/3.6.1/lib/python3.6/site-packages/numpy/lib/npyio.py", line 431, in load "Failed to interpret file %s as a pickle" % repr(file)) OSError: Failed to interpret file 'sample/gen_epoch_500.npz' as a pickle

    cvl@cvl-All-Series:~/workspace/3dpose_gan$ ls sample/gen_epoch_500.npz sample/gen_epoch_500.npz cvl@cvl-All-Series:~/workspace/3dpose_gan$ pip install -U pickle Collecting pickle Could not find a version that satisfies the requirement pickle (from versions: ) No matching distribution found for pickle cvl@cvl-All-Series:~/workspace/3dpose_gan$ pip install -U scipy Requirement already up-to-date: scipy in /home/cvl/.pyenv/versions/3.6.1/lib/python3.6/site-packages Requirement already up-to-date: numpy>=1.8.2 in /home/cvl/.pyenv/versions/3.6.1/lib/python3.6/site-packages (from scipy) cvl@cvl-All-Series:~/workspace/3dpose_gan$ python -V Python 3.6.1

    opened by dotchang 5
  • Cannot download gen_epoch_500.npz

    Cannot download gen_epoch_500.npz

    It seems the git lfs has a small quota, which has been exceeded. Could you provided another way to get the gen_epoch_500.npz?

    batch response: This repository is over its data quota. Purchase more data packs to restore access. error: failed to fetch some objects from 'https://github.com/DwangoMediaVillage/3dpose_gan.git/info/lfs'

    opened by horefice 2
  • Typo in running command

    Typo in running command

    Thanks for sharing this work. When running the demo, I guess the right command is

    python bin/demo.py sample/image.png --lift_model sample/gen_epoch_500.npz --model2d pose_iter_440000.caffemodel --proto2d openpose_pose_coco.prototxt
    

    One argument is model2d instead of model.

    opened by sunshineatnoon 1
  • Enable to train 3D pose for Human3.6M

    Enable to train 3D pose for Human3.6M

    Human3.6M

    • [x] unsupervised learning (ground truth 2d pose as input)
    • [x] supervised learning (ground truth 2d pose as input)
    • [x] evaluation of estimated 3d pose
    opened by yasunorikudo 1
  • cv2.error, ver problem?

    cv2.error, ver problem?

    When "run" demo the following error will be output Module versions are consistent.

    [libprotobuf ERROR /io/opencv/3rdparty/protobuf/src/google/protobuf/text_format.cc:288] Error parsing text-format opencv_caffe.NetParameter: 7:1: Expected identifier, got: <
    Traceback (most recent call last):
      File "bin/demo.py", line 199, in <module>
        main(args)
      File "bin/demo.py", line 163, in main
        points = OpenPose(args).predict(args, frame)
      File "bin/demo.py", line 108, in __init__
        self.net = cv.dnn.readNetFromCaffe(args.proto2d, args.model2d)
    cv2.error: OpenCV(3.4.5) /io/opencv/modules/dnn/src/caffe/caffe_io.cpp:1151: error: (-2:Unspecified error) FAILED: ReadProtoFromTextFile(param_file, param). Failed to parse NetParameter file: sample/openpose_pose_coco.prototxt in function 'ReadNetParamsFromTextFileOrDie'
    
    

    Do you have any advice?

    opened by nezumitti 0
  • cannot import name 'pose'

    cannot import name 'pose'

    When "run" demo the following error will be output Module versions are consistent.

    Traceback (most recent call last): File "bin/demo.py", line 9, in import evaluation_util File "/home/usr/download/3dpose_gan-master/bin/evaluation_util.py", line 10, in import projection_gan File "/usr/local/lib/python3.6/site-packages/projection_gan-0.0.1-py3.6.egg/projection_gan/init.py", line 1, in ImportError: cannot import name 'pose'

    Do you have any advice?

    opened by nezumitti 0
  • pixel coordinates or image plane coordinates??

    pixel coordinates or image plane coordinates??

    Hi,Excuse me, I have a little problem about some details in your code.

    In H3.6m dataset you used 3D data and real camera projection to generate 2D data, so the generated 2D points data are pixel coordinates instead of image plane coordinates because of multiplying the parameter f in the camera.

    So my problem is whether I need to divide f when normalizing the 2D points in order to get the image plane coordinates.The reason for this is because the assumption is that the orthogonal projection, that is, the 2D coordinates generated by the generator, are the image plane coordinates,while the input are pixel coordinates, which I think it will increase the error.

    Thanks

    opened by toBnumber1 0
  • Dimension mismatch when train MPII dataset?

    Dimension mismatch when train MPII dataset?

    How to train using MPII only 2D dataset(mpii_poses.npy)? Is it the same with H36M? When I train using MPII dataset, I got this error: Expect: x0.shape == x1.shape Actual: (16, 17) != (16, 0, 51)

    opened by JasOlean 0
  • Extending 3dpose_gan to posenet ?

    Extending 3dpose_gan to posenet ?

    Hi is it possible to extend this to posenet (which does the 2d joint detection? ) It has the following keypoint configuration

    'nose', 0 'leftEye', 1 'rightEye', 2 'leftEar', 3 'rightEar', 4 'leftShoulder', 5 'rightShoulder', 6 'leftElbow', 7 'rightElbow', 8 'leftWrist', 9 'rightWrist', 10 'leftHip', 11 'rightHip', 12 'leftKnee', ' 13 rightKnee', 14 'leftAnkle', 15 'rightAnkle' 16

    opened by timtensor 2
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022
Authors implementation of LieTransformer: Equivariant Self-Attention for Lie Groups

LieTransformer This repository contains the implementation of the LieTransformer used for experiments in the paper LieTransformer: Equivariant self-at

null 35 Oct 18, 2022
PyTorch Implementation of the SuRP algorithm by the authors of the AISTATS 2022 paper "An Information-Theoretic Justification for Model Pruning"

PyTorch Implementation of the SuRP algorithm by the authors of the AISTATS 2022 paper "An Information-Theoretic Justification for Model Pruning".

Berivan Isik 8 Dec 8, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
This repo is a PyTorch implementation for Paper "Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds"

Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds This repository is a PyTorch implementation for paper: Uns

Kaizhi Yang 42 Dec 9, 2022
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

Aymen Mir 66 Dec 21, 2022
Classic Papers for Beginners and Impact Scope for Authors.

There have been billions of academic papers around the world. However, maybe only 0.0...01% among them are valuable or are worth reading. Since our limited life has never been forever, TopPaper provide a Top Academic Paper Chart for beginners and reseachers to take one step faster.

Qiulin Zhang 228 Dec 18, 2022
Text mining project; Using distilBERT to predict authors in the classification task authorship attribution.

DistilBERT-Text-mining-authorship-attribution Dataset used: https://www.kaggle.com/azimulh/tweets-data-for-authorship-attribution-modelling/version/2

null 1 Jan 13, 2022
Implementation of "JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargeting"

JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargeting Pytorch implementation for the paper "JOKR: Joint Keypoint Repres

null 45 Dec 25, 2022
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
Code for "Single-view robot pose and joint angle estimation via render & compare", CVPR 2021 (Oral).

Single-view robot pose and joint angle estimation via render & compare Yann Labbé, Justin Carpentier, Mathieu Aubry, Josef Sivic CVPR: Conference on C

Yann Labbé 51 Oct 14, 2022
Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments Paper: arXiv (ICRA 2021) Video : https://youtu.be/CC

Sachini Herath 68 Jan 3, 2023
SMPL-X: A new joint 3D model of the human body, face and hands together

SMPL-X: A new joint 3D model of the human body, face and hands together [Paper Page] [Paper] [Supp. Mat.] Table of Contents License Description News I

Vassilis Choutas 1k Jan 9, 2023
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Pytorch implementation for A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose

A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose Paper | Website | Data A-NeRF: Articulated Neural Radiance F

Shih-Yang Su 172 Dec 22, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
Official PyTorch implementation of "IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos", CVPRW 2021

IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos Introduction This repo is official PyTorch implementatio

Gyeongsik Moon 29 Sep 24, 2022
Official Pytorch implementation of "Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video", CVPR 2021

TCMR: Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video Qualtitative result Paper teaser video Introduction This r

Hongsuk Choi 215 Jan 6, 2023
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 363 Dec 28, 2022