Exploring Versatile Prior for Human Motion via Motion Frequency Guidance (3DV2021)

Related tags

Deep Learning motion-prior-reconstruction
Overview

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance

This is the codebase for video-based human motion reconstruction in human-motion-prior.

[Video Demo] [Paper]

Installation

Requirements

  • Python 3.6
  • PyTorch 1.1.0

Because this project is based on our pretrained human motion prior, please clone the prior repository and this repository as follows:

git clone https://github.com/JchenXu/human-motion-prior.git human_motion_prior                                                                                          
git clone https://github.com/JchenXu/motion-prior-reconstruction.git

and run the following command to install the dependencies:

pip install -r requirements.txt

Data Preparation

Please download the required data (i.e., our pre-trained prior model and SMPL model parameters) here, and then, uncompress and put it in data/mp_data.

Then, refer to this for data generation, and put all data files in data/mp_db.

The whole data directory is like:

motion-prior-reconstruction/data
├── mp_data
│   ├── ...
|   └── ...
|
├── mp_db
    ├── 3dpw_train_db.pt
    └── insta_train_db.h5
    └── ...

Training

Run the commands below to start training:

export PYTHONPATH=../human_motion_prior
python train.py --cfg configs/config_3dpw.yaml

Evaluation

Modify the trained checkpoint in configs to evaluation your trained model. Then, run the commands below to start evaluation:

export PYTHONPATH=../human_motion_prior
python eval.py --cfg configs/config_3dpw.yaml

Also, we provide our pre-trained checkpoint here.

Citation

@inproceedings{human_motion_prior,
  title = {Exploring Versatile Prior for Human Motion via Motion Frequency Guidance},
  author = {Jiachen Xu, Min Wang, Jingyu Gong, Wentao Liu, Chen Qian, Yuan Xie, Lizhuang Ma},
  booktitle = {2021 international conference on 3D vision (3DV)},
  year = {2021}
}

Acknowledgement

We thank the authors of VIBE for their released code, and this base codes are largely borrowed from them.

You might also like...
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Exploring Image Deblurring via Encoded Blur Kernel Space About the project We introduce a method to encode the blur operators of an arbitrary dataset

VinAI Research 118 Dec 19, 2022
Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer"

TSOD Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer" Usage For training, open train_test, run p

Jinming Su 2 Dec 23, 2021
[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis [arxiv|pdf|v

Yinan He 78 Dec 22, 2022
Totally Versatile Miscellanea for Pytorch

Totally Versatile Miscellania for PyTorch Thomas Viehmann [email protected] This repository collects various things I have implmented for PyTorch Laye

Thomas Viehmann 428 Dec 28, 2022
The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.
The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.

CrossFormer This repository is the code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention. Introduction Existin

cheerss 238 Jan 6, 2023
Versatile Generative Language Model
Versatile Generative Language Model

Versatile Generative Language Model This is the implementation of the paper: Exploring Versatile Generative Language Model Via Parameter-Efficient Tra

Zhaojiang Lin 17 Dec 2, 2022
X-modaler is a versatile and high-performance codebase for cross-modal analytics.
X-modaler is a versatile and high-performance codebase for cross-modal analytics.

X-modaler X-modaler is a versatile and high-performance codebase for cross-modal analytics. This codebase unifies comprehensive high-quality modules i

null 910 Dec 28, 2022
Learning Versatile Neural Architectures by Propagating Network Codes
Learning Versatile Neural Architectures by Propagating Network Codes

Learning Versatile Neural Architectures by Propagating Network Codes Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang,

Mingyu Ding 36 Dec 6, 2022
[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

null 129 Jan 4, 2023
Comments
  • Reproduction of the Table2 in H3.6M evaluation

    Reproduction of the Table2 in H3.6M evaluation

    I use this link SMPL from SMPLify-X to access H3.6M data , and I just use this to reproduce the result in Table 2 in H3.6M evaluation part. I get this result

    MPJPE | PA-MPJPE | ACCEL 73.84 | 48.76 | 28.06

    in original paper, it is

    MPJPE | PA-MPJPE | ACCEL 65.6 | 41.0 | 13.9

    I have been confused. Could you help me? I just want to reproduce this great work. Thanks.

    opened by amituofo1996 2
Owner
Jiachen Xu
Jiachen Xu
GitHub
[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

Versatile Multi-Modal Pre-Training for Human-Centric Perception Fangzhou Hong1  Liang Pan1  Zhongang Cai1,2,3  Ziwei Liu1* 1S-Lab, Nanyang Technologic

Fangzhou Hong 96 Jan 3, 2023
Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021.

Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021. Figure 1: In the process of motion capture (mocap), some joints or even the whole human

Shinny cui 3 Oct 31, 2022
Reviving Iterative Training with Mask Guidance for Interactive Segmentation

This repository provides the source code for training and testing state-of-the-art click-based interactive segmentation models with the official PyTorch implementation

Visual Understanding Lab @ Samsung AI Center Moscow 406 Jan 1, 2023
Learning Calibrated-Guidance for Object Detection in Aerial Images

Learning Calibrated-Guidance for Object Detection in Aerial Images arxiv We propose a simple yet effective Calibrated-Guidance (CG) scheme to enhance

null 51 Sep 22, 2022
dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ)

dualFace dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ) We provide python implementations for our CVM 2021 paper "dualFac

Haoran XIE 46 Nov 10, 2022
Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing

EGFNet Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing Dataset and Results Test maps: 百度网盘 提取码：zust Citation @ARTICLE{ author={Zhou,

ShaohuaDong 10 Dec 8, 2022
PyTorch implementation of SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching

SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching This is the official PyTorch implementation of SMODICE: Versatile Offline I

Jason Ma 14 Aug 30, 2022
[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

pytorch-deep-video-prior (DVP) Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior TensorFlo

Yazhou XING 90 Oct 19, 2022
This repository contains the code for the paper "Hierarchical Motion Understanding via Motion Programs"

Hierarchical Motion Understanding via Motion Programs (CVPR 2021) This repository contains the official implementation of: Hierarchical Motion Underst

Sumith Kulal 40 Dec 5, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 6, 2023