This repository contains a pytorch implementation of "StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision".

Last update: Dec 9, 2022

Related tags

Overview

StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision

| Project Page | Paper |

This repository contains a pytorch implementation of "StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision (CVPR 2021)".
Authors: Yang Hong, Juyong Zhang, Boyi Jiang, Yudong Guo, Ligang Liu and Hujun Bao.

Requirements

Python 3
Pytorch (<=1.4.0, some compatibility issues may occur in higher versions of pytorch)
tqdm
opencv-python
scikit-image
openmesh

for building evaluation data

pybind11,we recommend "pip install pybind11[global]" for installation.
gcc
cmake

Run the following code to install all pip packages:

pip install -r requirements.txt

Building Evaluation Data

Preliminary

Run the following script to compile & generate the relevant python module, which is used to render left/right color/depth/mask images from the textured/colored mesh.

cd GenEvalData
bash build.sh
cd ..

Usage

#demo, for textured mesh
python GenEvalData.py \
--tex_mesh_path="TempData/SampleData/rp_dennis_posed_004_100k.obj" \
--tex_img_path="TempData/SampleData/rp_dennis_posed_004_dif_2k.jpg" \
--save_dir="./TempData/TexMesh" \
--save_postfix="tex"

#demo, for colored mesh
python GenEvalData.py \
--color_mesh_path="TempData/SampleData/normalized_mesh_0089.off" \
--save_dir="./TempData/ColorMesh" \
--save_postfix="color"

These samples are from renderpeople and BUFF dataset.
Note: the mesh used for rendering needs to be located in a specific bounding box.

Inference

Preliminary

Run the following script to compile & generate deformable convolution from AANet.
```
cd AANetPlusFeature/deform_conv
bash build.sh
cd ../..
```
Download the trained model and mv to the "Models" folder.
Generate evalution data with aboved "Building Evaluation Data", or capture real data by ZED Camera (we test on ZED camera v1).
Note: rectifying left/right images is required before using ZED camera.

Demo

bash eval.sh

The reconsturction result will be saved to "Results" folder.
Note: At least 10GB GPU memory is recommended to run StereoPIFu model.

Citation

@inproceedings{yang2021stereopifu,
  author    = {Yang Hong and Juyong Zhang and Boyi Jiang and Yudong Guo and Ligang Liu and Hujun Bao},
  title     = {StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision},
  booktitle = {{IEEE/CVF} Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2021}
}

Contact

If you have questions, please contact [email protected].

Comments

BUFF dataset evaluation

Hi,

thanks for your amazing work. Could you please share the code snippet for the rendering and evaluation of the BUFF dataset? Many thanks in advance!

opened by MoyGcc 3
Running eval.h with less than 10gb?

Is it possible to run the provided eval.sh sample with a single GPU that has less than 10gb? Changing some parameters in the code or using a different resolution for the images? or using a combo of 2 GPUs that equate to 10gb?

opened by guayabas 2
How can I use Animated Model from Renderpeople?

First of all, thanks for sharing such a great research. I am learning so much from this work.

I wanted to use 3D animated people as an input, so I tried using obj files per frame, exported from 3D animated people (Renderpeople dataset).

However I got Segmentation fault (core dumped) error while generating evaluation data. It seems like error occured at RenderUtils.render_tex_mesh() part.

So I was wandering if you had solutions for this... Any answers would be so grateful. Thank you!

opened by hshlego 0
The code for 1ualitative error visualization

Hi, thanks for your excellent work and open-sourcing your code. I have successfully reproduced your code. However, I would like to implement some cool visualization results. Can you share the code for visualizing the qualitativee error of Figure 5 in your paper?

opened by sunjc0306 3
How is the unit determined in the BUFF dataset

The unit of evaluation metrics reported in your paper is cm. I would like to know how you obtain this unit. In another word, what is the relationship between the vertex coordinates of the buff dataset and the centimeter unit? Could you share your script for this process?

opened by sunjc0306 4
Any plan to publish train script?

You did some great research!

I also want to train the network through data similar to the data used in the thesis.

Do you have any plans to publish training or training data preprocessing code?

opened by bell-one 3

This repository contains a pytorch implementation of "StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision".

Related tags

Overview

StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision

| Project Page | Paper |

Requirements

Building Evaluation Data

Preliminary

Usage

Inference

Preliminary

Demo

Citation

Contact

Comments

BUFF dataset evaluation

Running eval.h with less than 10gb?

How can I use Animated Model from Renderpeople?

The code for 1ualitative error visualization

How is the unit determined in the BUFF dataset

Any plan to publish train script?

Owner

This repository contains the PyTorch implementation of the paper STaCK: Sentence Ordering with Temporal Commonsense Knowledge appearing at EMNLP 2021.

RGBD-Net - This repository contains a pytorch lightning implementation for the 3DV 2021 RGBD-Net paper.

This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing

This repository contains PyTorch code for Robust Vision Transformers.

This repository contains PyTorch models for SpecTr (Spectral Transformer).

An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

This repository contains the implementation of Deep Detail Enhancment for Any Garment proposed in Eurographics 2021

This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effects in Video."

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

This repository contains the implementation of the paper: "Towards Frequency-Based Explanation for Robust CNN"

This repository contains numerical implementation for the paper Intertemporal Pricing under Reference Effects: Integrating Reference Effects and Consumer Heterogeneity.

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).

This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

This repository contains the code for our fast polygonal building extraction from overhead images pipeline.

This repository contains the code for the paper "Hierarchical Motion Understanding via Motion Programs"

This repository contains all the code and materials distributed in the 2021 Q-Programming Summer of Qode.

This repository contains the code, data, and models of the paper titled "XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages" published in Findings of the Association for Computational Linguistics: ACL 2021.