Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval PyTorch
This is the PyTorch implementation of Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval.
Abstract:
We present Retrieve in Style (RIS), an unsupervised framework for fine-grained facial feature transfer and retrieval on real images Recent work shows that it is possible to learn a catalog that allows local semantic transfers of facial features on generated images by capitalizing on the disentanglement property of the StyleGAN latent space. RIS improves existing art on:
- feature disentanglement and allows for challenging transfers (\ie, hair and pose) that were not shown possible in SoTA methods.
- eliminating the needs for per-image hyperparameter tuning, and for computing a catalog over a large batch of images.
- enabling face retrieval using the proposed facial features (\eg, eyes), and to our best knowledge, is the first work to retrieve face images at the fine-grained level.
- robustness and natural application to real images. Our qualitative and quantitative analyses show RIS achieves both high-fidelity feature transfers and accurate fine-grained retrievals on real images. We discuss the responsible application of RIS.
Dependency
Our codebase is based off stylegan2 by rosalinity.
conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=<CUDA_VERSION>
pip install tqdm gdown scikit-learn scipy lpips dlib opencv-python
How to use
Everything to get started is in the colab notebook.
Citation
If you use this code or ideas from our paper, please cite our paper:
@article{chong2021retrieve,
title={Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval},
author={Chong, Min Jin and Chu, Wen-Sheng and Kumar, Abhishek},
journal={arXiv preprint arXiv:2107.06256},
year={2021}
}
{"mode":"full","isActive":false}
Acknowledgments
This code borrows from StyleGAN2 by rosalinity, Editing in Style, StyleClip, PTI. Encoder used is borrowed directly from encoder4editing.
54 Dec 8, 2022
19 May 4, 2022
80 Dec 30, 2022
7 Jul 16, 2020
4 Sep 18, 2022
770 Jan 2, 2023
9 Oct 15, 2022
142 Jan 8, 2023
148 Dec 30, 2022
131 Dec 19, 2022
16 Nov 4, 2020
4 Nov 3, 2022
5.8k Dec 31, 2022
1 Feb 7, 2022
123 Dec 27, 2022
75 Dec 16, 2022
38 Oct 14, 2022
49 Oct 13, 2022
10.9k Jan 2, 2023