Exploring Versatile Prior for Human Motion via Motion Frequency Guidance (3DV2021)

Related tags

Deep Learning human-motion-prior
Overview

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance

[Video Demo] [Paper]

Installation

Requirements

  • Python 3.6
  • PyTorch 1.1.0

Please clone this repository and run the following command:

pip install -r requirements.txt

Then, download some checkpoints and human body models, and place them in the human_motion_prior/models.

Specifically,

  • Download the pretrained VPoser v1.0 model (2.5 MB) here.
  • Download the SMPLH model here.
  • Download the SMPL model here for MALE and FEMALE and here for NEUTRAL.
  • Download the regressor weights files 1) J_regressor_h36m.npy and 2) J_regressor_extra.npy here.

The models tree is like:

human_motion_prior/models
├── smpl
│   ├── SMPL_FEMALE.pkl
│   └── SMPL_MALE.pkl
│   └── SMPL_NEUTRAL.pkl
│   └── J_regressor_h36m.npy
│   └── J_regressor_extra.npy
├── smplh
│   ├── SMPLH_FEMALE_AMASS.npz
│   └── SMPLH_MALE_AMASS.npz
|   └── SMPLH_MEUTRAL_AMASS.npz
├── pre_trained
    └── vposer_v1_0

Data Pre-process

Follow the instruction here.

Training

cd human_motion_prior/train
sh run_script.sh 4

We train our human motion prior on 4 GTX 1080Ti gpus with batch size 15 per gpu.

You can modify the human_motion_prior/train/motion_prior_defaults.ini for different training setting.

Inference

You can use the motion prior trained on AMASS to evaluate the VAE reconstruction loss on unseen 3DPW dataset as follows:

cd human_motion_prior/test
export PYTHONPATH=../../
python test_3dpw.py

Citation

@inproceedings{human_motion_prior,
  title = {Exploring Versatile Prior for Human Motion via Motion Frequency Guidance},
  author = {Jiachen Xu, Min Wang, Jingyu Gong, Wentao Liu, Chen Qian, Yuan Xie, Lizhuang Ma},
  booktitle = {2021 international conference on 3D vision (3DV)},
  year = {2021}
}

Acknowledgments

We thank the authors of VPoser for their released code.

You might also like...
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Exploring Image Deblurring via Encoded Blur Kernel Space About the project We introduce a method to encode the blur operators of an arbitrary dataset

VinAI Research 118 Dec 19, 2022
Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer"

TSOD Code for the ICME 2021 paper "Exploring Driving-Aware Salient Object Detection via Knowledge Transfer" Usage For training, open train_test, run p

Jinming Su 2 Dec 23, 2021
[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis [arxiv|pdf|v

Yinan He 78 Dec 22, 2022
Totally Versatile Miscellanea for Pytorch

Totally Versatile Miscellania for PyTorch Thomas Viehmann [email protected] This repository collects various things I have implmented for PyTorch Laye

Thomas Viehmann 428 Dec 28, 2022
The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.
The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.

CrossFormer This repository is the code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention. Introduction Existin

cheerss 238 Jan 6, 2023
Versatile Generative Language Model
Versatile Generative Language Model

Versatile Generative Language Model This is the implementation of the paper: Exploring Versatile Generative Language Model Via Parameter-Efficient Tra

Zhaojiang Lin 17 Dec 2, 2022
X-modaler is a versatile and high-performance codebase for cross-modal analytics.
X-modaler is a versatile and high-performance codebase for cross-modal analytics.

X-modaler X-modaler is a versatile and high-performance codebase for cross-modal analytics. This codebase unifies comprehensive high-quality modules i

null 910 Dec 28, 2022
Learning Versatile Neural Architectures by Propagating Network Codes
Learning Versatile Neural Architectures by Propagating Network Codes

Learning Versatile Neural Architectures by Propagating Network Codes Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang,

Mingyu Ding 36 Dec 6, 2022
[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

null 129 Jan 4, 2023
Comments
  • About Training Data Problem

    About Training Data Problem

    Hi, Great work! When I prepare data and start trainning , one problem happen.

    (human_motion_prior) xiaofeng@xiaofeng-Precision-Tower-7910:/media/xiaofeng/E/code/human-motion-prior-main/human_motion_prior/train$ sh run_script.sh 1 tensorboard --logdir=logs/motion_prior/summaries Torch Version: 1.1.0 1 CUDAs available! Training with cuda [TITAN Xp] Base dataset_dir is ../data/preprocess/amass_smpl_30fps_128frame.npz self.data.shape[0] 35429 self.p3ds.shape[0] 35338 Traceback (most recent call last): File "train_motion_prior.py", line 18, in run_motion_prior_trainer(ps) File "/media/xiaofeng/E/code/human-motion-prior-main/human_motion_prior/train/motion_prior.py", line 651, in run_motion_prior_trainer vp_trainer = MotionPriorTrainer(ps.work_dir, ps) File "/media/xiaofeng/E/code/human-motion-prior-main/human_motion_prior/train/motion_prior.py", line 307, in init ds_train = AMASSSeqDataset(data_dir=ps.dataset_dir, is_train=True) File "/media/xiaofeng/E/code/human-motion-prior-main/human_motion_prior/data/seq_dataloader.py", line 20, in init assert self.data.shape[0] == self.p3ds.shape[0], 'training data length should be same to the p3ds'

    AssertionError: training data length should be same to the p3ds

    Traceback (most recent call last): File "/home/xiaofeng/anaconda3/envs/human_motion_prior/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/xiaofeng/anaconda3/envs/human_motion_prior/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/xiaofeng/anaconda3/envs/human_motion_prior/lib/python3.6/site-packages/torch/distributed/launch.py", line 235, in main() File "/home/xiaofeng/anaconda3/envs/human_motion_prior/lib/python3.6/site-packages/torch/distributed/launch.py", line 231, in main cmd=process.args) subprocess.CalledProcessError: Command '['/home/xiaofeng/anaconda3/envs/human_motion_prior/bin/python', '-u', 'train_motion_prior.py', '--local_rank=0']' returned non-zero exit status 1.

    It seems pose data length is different with joint data length. how can i fix this problem,Thanks

    opened by amituofo1996 2
Owner
Jiachen Xu
Jiachen Xu
GitHub
[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

Versatile Multi-Modal Pre-Training for Human-Centric Perception Fangzhou Hong1  Liang Pan1  Zhongang Cai1,2,3  Ziwei Liu1* 1S-Lab, Nanyang Technologic

Fangzhou Hong 96 Jan 3, 2023
Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021.

Code of the paper "Deep Human Dynamics Prior" in ACM MM 2021. Figure 1: In the process of motion capture (mocap), some joints or even the whole human

Shinny cui 3 Oct 31, 2022
Reviving Iterative Training with Mask Guidance for Interactive Segmentation

This repository provides the source code for training and testing state-of-the-art click-based interactive segmentation models with the official PyTorch implementation

Visual Understanding Lab @ Samsung AI Center Moscow 406 Jan 1, 2023
Learning Calibrated-Guidance for Object Detection in Aerial Images

Learning Calibrated-Guidance for Object Detection in Aerial Images arxiv We propose a simple yet effective Calibrated-Guidance (CG) scheme to enhance

null 51 Sep 22, 2022
dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ)

dualFace dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ) We provide python implementations for our CVM 2021 paper "dualFac

Haoran XIE 46 Nov 10, 2022
Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing

EGFNet Edge-aware Guidance Fusion Network for RGB-Thermal Scene Parsing Dataset and Results Test maps: 百度网盘 提取码：zust Citation @ARTICLE{ author={Zhou,

ShaohuaDong 10 Dec 8, 2022
PyTorch implementation of SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching

SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching This is the official PyTorch implementation of SMODICE: Versatile Offline I

Jason Ma 14 Aug 30, 2022
[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

pytorch-deep-video-prior (DVP) Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior TensorFlo

Yazhou XING 90 Oct 19, 2022
This repository contains the code for the paper "Hierarchical Motion Understanding via Motion Programs"

Hierarchical Motion Understanding via Motion Programs (CVPR 2021) This repository contains the official implementation of: Hierarchical Motion Underst

Sumith Kulal 40 Dec 5, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 6, 2023