HomoInterpGAN
Homomorphic Latent Space Interpolation for Unpaired Image-to-image Translation (CVPR 2019, oral)
Installation
The implementation is based on pytorch. Our model is trained and tested on version 1.0.1.post2. Please install relevant packages based on your own environment.
All other required packages are listed in "requirements.txt". Please run
pip install -r requirements.txt
to install these packages.
Dataset
Download the "Align&Cropped Images" of the CelebA dataset. If the original link is unavailable, you can also download it here.
Training
Firstly, cd to the project directory and run
export PYTHONPATH=./:$PYTHONPATH
before executing any script.
To train a model on CelebA, please run
python run.py train --data_dir CELEBA_ALIGNED_DIR -sp checkpoints/CelebA -bs 128 -gpu 0,1,2,3
Key arguments
--data_dir: The path of the celeba_aligned images.
-sp: The trained model and logs, intermediate results are stored in this directory.
-bs: Batch size.
-gpu: The GPU index.
--attr: This specifies the target attributes. Note that we concatenate multiple attributes defined in CelebA as our grouped attribute. We use "@" to group multiple multiple attributes to a grouped one (e.g., Mouth_Slightly_Open@Smiling forms a "expression" attriute). We use "," to split different grouped attributes. See the default argument of "run.py" for details.
Testing
python run.py attribute_manipulation -mp checkpoints/CelebA -sp checkpoints/CelebA/test/Smiling --filter_target_attr Smiling -s 1 --branch_idx 0 --n_ref 5 -bs 8
This conducts attribute manipulation with reference samples selected in CelebA dataset. The reference samples are selected based on their attributes (--filter_target_attr), and the interpolation path should be chosen accordingly.
Key arguments:
-mp: the model path. The checkpoints of encoder, interpolator and decoder should be stored in this path.
-sp: the save path of the results.
--filter_target_attr: This specifies the attributes of the reference images. The attribute names can be found in "info/attribute_names.txt". We can specify one attribute (e.g., "Smiling") or several attributes (e.g., "Smiling@Mouth_Slightly_Open" will filter mouth open smiling reference images). To filter negative samples, add "NOT" as prefix to the attribute names, such as "NOTSmiling", "NOTSmiling@Mouth_Slightly_Open".
--branch_idx: This specifies the branch index of the interpolator. Each branch handles a group of attribute. Note that the physical meaning of each branch is specified by "--attr" during testing.
-s: The strength of the manipulation. Range of [0, 2] is suggested. If s>1, the effect is exaggerated.
-bs: the batch size of the testing images.
-n_ref: the number of images used as reference.
Testing on unaligned images
Note the the performance could degenerate if the testing image is not well aligned. Thus we also provide a tool for face alignment. Please place all your testing images to a folder (e.g., examples/original), then run
python facealign/align_all.py examples/original examples/aligned
to align testing images to an samples in CelebA. Then you can run manipulation by
python run.py attribute_manipulation -mp checkpoints/CelebA -sp checkpoints/CelebA/test/Smiling --filter_target_attr Smiling -s 1 --branch_idx 0 --n_ref 5 -bs 8 --test_folder examples/aligned
Note that an additional argument "--test_folder" is specified.
Pretrained model
We have also provided a pretrained model here. It is trained with default parameters. The meaning of each branch of the interpolator is listed bellow.
| Branch index | Grouped attribute | Corresponding labels on CelebA |
|---|---|---|
| 1 | Expression | Mouth_Slightly_Open, Smiling |
| 2 | Gender trait | Male, No_Beard, Mustache, Goatee, Sideburns |
| 3 | Hair color | Black_Hair, Blond_Hair, Brown_Hair, Gray_Hair |
| 4 | Hair style | Bald, Receding_Hairline, Bangs |
| 5 | Age | Young |
Updates
- Jun 17, 2019: It is observed that the face alignment tool is not perfect, and the results of "Testing on unaligned images" does not perform as well as results in CelebA dataset. To make the model less sensitive of the alignment issue, we add random shifting in center_crop during training. The shifting range can be controlled by "--random_crop_bias". We have updated the pretarined model by fine-tuning it with "random_crop_bias=10", which leads to better results in unaligned images.
Reference
Ying-Cong Chen, Xiaogang Xu, Zhuotao Tian, Jiaya Jia, "Homomorphic Latent Space Interpolation for Unpaired Image-to-image Translation" , Computer Vision and Pattern Recognition (CVPR), 2019 PDF
@inproceedings{chen2019Homomorphic,
title={Homomorphic Latent Space Interpolation for Unpaired Image-to-image Translation},
author={Chen, Ying-Cong and Xu, Xiaogang and Tian, Zhuotao and Jia, Jiaya},
booktitle={CVPR},
year={2019}
}
Contect
Please contact [email protected] if you have any question or suggestion.
191 Dec 31, 2022
172 Dec 22, 2022
150 Dec 7, 2022
55 Dec 6, 2022
58 Dec 24, 2022
67 Dec 21, 2022
1.3k Dec 29, 2022
7k Dec 30, 2022
38 Dec 15, 2022
135 Dec 30, 2022
100 Dec 6, 2022
1 Oct 28, 2022
4 Jun 17, 2022
2 Nov 21, 2022
3 Jul 8, 2022
86 Nov 16, 2022
10 Oct 28, 2022
37 Dec 30, 2022