PyTorch implementation of our ICCV 2021 paper Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer.

Related tags

Deep Learning pytorch style-transfer meshes 3d
Overview

Unsupervised_IEPGAN

This is the PyTorch implementation of our ICCV 2021 paper Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer.
Haoyu Chen, Hao Tang, Henglin Shi, Wei Peng, Nicu Sebe, Guoying Zhao.

Citation

If you use our code or paper, please consider citing:

@inproceedings{chen2021IEPGAN,
  title={Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer},
  author={Chen, Haoyu and Tang, Hao and Henglin, Shi and Peng, Wei and Sebe, Nicu and Zhao, Guoying},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2021}
}

Dependencies

Requirements:

  • python3.6
  • numpy
  • pytorch==1.1.0 and above
  • trimesh

Dataset preparation

Please download FAUST data from FAUST link, and DFAUST dataset from DFAUST link for training the model.

Usage

The usage of our code is easy, just run the code below.

bash run.sh

Your can change the hyperparameters according to your GPU settings and needs in the run.sh file:

python  train.py --batch 4 ./dataset_3d/FAUST/  --edge_loss_setting 'rec_rec' --experi_path 'full_adap_GIH_Lap_rec' --server 'local' --rec_epoch 1200 --geo_epoch 1600 --iter 2000 --geoloss 0.005 --n_crop 1 --ref_crop 2 --sampling_number 120 --limb_n 4 --limb_sampling 600  --sampling_pattern 'adaptive'

Acknowledgement

Part of our code is based on

3D transfer: NPT，

GAN framework: SWAP-Autoencoder (unofficial implementation) and

GIH implementation: LIMP. Many thanks!

License

MIT-2.0 License

Comments
  • FAUST_DATA_with_GIH

    FAUST_DATA_with_GIH

    I could not understand how to get the data that FAUST_DATA_with_GIH requires.

    Specifically, it searches for obj files: pose_mesh_path = self.path+'tr_reg_'+str(pose_mesh_idx).zfill(3)+'.obj'. However, the FAUST dataset supplies ply files with this prefix (e.g. tr_scan_030.ply).

    Furthermore, gt_path = self.path_GIH_gt+'tr_reg_'+str(identity_mesh_idx).zfill(3)+'.pkl' expects pkl file which I can't understand how to produce. The LIMP repository produces folders (not pkl files) with this prefix e.g. tr_reg_099_2000. Each folder contains two files: 1. mesh.triv, 2. mesh.vert.

    Can you please explain how to get the pkl and obj files? Alternatively, I read in another issue that you planned to upload the preprocessed data to the cloud 3 months ago. Would it be possible for you to upload the data?

    Thanks, Yoni

    opened by ykasten 3
  • What is the

    What is the "SMPL" dataset?

    Hi, Thank you very much for sharing the code and for your paper! I'm trying to launch some training but there is the "dataset_rec" I'm not sure how to build. I guess it is made of 16 SMPL individuals where you "swap" identity and pose SMPL parameters?

    As far as I understood, it is linked to this sentence "For the first 2 × 104 iterations, only reconstruction is conducted to stabilize the GAN and avoid local minima, where the pair of reconstruction adversarial loss is used with raw SMPL meshes." in the paper right?

    Where did those parameters come from? It wouldn't be too hard to generate some with AMASS but I would like to know which part was used in order to have the same diversity. If you could share a .zip or something it would be awesome, but I would understand that you're out of time right now :smile:

    opened by daidedou 2
  • Problem about data pre-processing

    Problem about data pre-processing

    Thank you for this wonderful work! I have one questions about data pre-processing. When I use 'generate_geodesic_distance.py' to convert the OBj file you provided to GIH, I encountered the following error: ` f = torch.solve(U, B)[0] Traceback (most recent call last):

    File "GIH_calculating/generate_geodesic_distance.py", line 39, in Dg_r, grad, div, W, S, C = distance_GIH(mesh_points, mesh_faces)

    File "/mnt/16T/sxk/Unsupervised_IEPGAN-main/GIH_calculating/utils_distance.py", line 122, in distance_GIH D = _geodesics_in_heat(grad,div,W[0],A,t)

    File "/mnt/16T/sxk/Unsupervised_IEPGAN-main/GIH_calculating/utils_distance.py", line 109, in _geodesics_in_heat f = torch.solve(U, B)[0]

    RuntimeError: solve_cuda: The diagonal element 1 is zero, the solve could not be completed because the input matrix is singular.` Can you explain it? Thank you again for your contribution. Thanks, xiaokun

    opened by 2017211801 1
  • Generalization problem

    Generalization problem

    Hi, Thanks for the inspiring work! I have two questions about this work. Could this method adapt to meshes with different number of vertices in both training and testing process? Also, would you release the pre-trained model for testing? Thanks!

    opened by jinmelo 3
  • The shuffle of the vertices

    The shuffle of the vertices

    Hi, thank you for your work.

    I noticed that you remove the shuffle of the vertices of the identity and pose meshes which added in NPT. Have you tried to add the shuffle in your work?

    opened by didiya5154 3
Owner
A researcher in Oulu, Finland
null
GitHub
Code for our NeurIPS 2021 paper 'Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation'

Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation (NeurIPS 2021) Code for our NeurIPS 2021 paper 'Exploiting the Intri

Shiqi Yang 53 Dec 25, 2022
Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data (CVPR 2022) Potentials of primitive shapes f

null 31 Sep 27, 2022
Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images"

GANInversion_with_ConsecutiveImgs Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images" https://a

QingyangXu 38 Dec 7, 2022
(ICCV 2021) Official code of "Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing."

Dressing in Order (DiOr) ?? [Paper] ?? [Webpage] ?? [Running this code] The official implementation of "Dressing in Order: Recurrent Person Image Gene

Aiyu Cui 277 Dec 28, 2022
PyTorch implementation of paper: AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer, ICCV 2021.

AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer [Paper] [PyTorch Implementation] [Paddle Implementation] Overview This reposit

null 148 Dec 30, 2022
Official pytorch code for SSC-GAN: Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation(ICCV 2021)

SSC-GAN_repo Pytorch implementation for 'Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation'.PDF SSC-GAN:Sem

tyty 4 Aug 28, 2022
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

Ran Cheng 4 Dec 15, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

GANs for Biological Image Synthesis This codes implements the ICCV-2017 paper "GANs for Biological Image Synthesis". The paper and its supplementary m

Anton Osokin 95 Nov 25, 2022
Code for "The Intrinsic Dimension of Images and Its Impact on Learning" - ICLR 2021 Spotlight

dimensions Estimating the instrinsic dimensionality of image datasets Code for: The Intrinsic Dimensionaity of Images and Its Impact On Learning - Phi

Phil Pope 41 Dec 10, 2022
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

Ren Tianhe 49 Nov 10, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 225 Dec 29, 2022
PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

Saim Wani 4 May 8, 2022
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
Repository for the paper "PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation", CVPR 2021.

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation Code repository for the paper: PoseAug: A Differentiable Pose Augme

Pyjcsx 328 Dec 17, 2022
《Lerning n Intrinsic Grment Spce for Interctive Authoring of Grment Animtion》

Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation Overview This is the demo code for training a motion invariant enco

YuanBo 213 Dec 14, 2022
Intrinsic Image Harmonization

Intrinsic Image Harmonization [Paper] Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, Bing Zheng Here we provide PyTorch implementation and the

VISION @ OUC 44 Dec 21, 2022
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Tengfei Wang 110 Dec 20, 2022
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Tengfei Wang 110 Dec 20, 2022