Official PyTorch Implementation for InfoSwap: Information Bottleneck Disentanglement for Identity Swapping

Grace Hešeri

Last update: Dec 20, 2022

Related tags

Deep Learning InfoSwap-master

Overview

InfoSwap: Information Bottleneck Disentanglement for Identity Swapping

Code usage

Please check out the user manual page.

Paper

Gege Gao, Huaibo Huang, Chaoyou Fu, Zhaoyang Li, Ran He, "Information Bottleneck Disentanglement for Identity Swapping", CVPR 2021

Results Across Large Gaps:

Results of 1024x1024 Pixels:

Results in Film Scenes:

Citation

If you find this code useful for your research, please cite our paper:

@InProceedings{Gao_2021_CVPR,
    author    = {Gao, Gege and Huang, Huaibo and Fu, Chaoyou and Li, Zhaoyang and He, Ran},
    title     = {Information Bottleneck Disentanglement for Identity Swapping},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {3404-3413}
}

Comments

太棒的思想了，看源码有个问题
iib.py里面 191行，标准差为什么取的是均值呢？不太懂 m_s = torch.mean(Rs, dim=0) # [C, H, W] std_s = torch.mean(Rs, dim=0) Rs_params.append([m_s, std_s]) eps_s = torch.randn(size=Rt.shape).to(Rt.device) * std_s + m_s feat_t = Rt * (1. - lambda_t) + lambda_t * eps_s Xt_feats.append(feat_t) # only related with lambda

m_t = torch.mean(Rt, dim=0) # [C, H, W] std_t = torch.mean(Rt, dim=0) Rt_params.append([m_t, std_t]) eps_t = torch.randn(size=Rs.shape).to(Rs.device) * std_t + m_t feat_s = Rs * (1. - lambda_s) + lambda_s * eps_t Xs_feats.append(feat_s) # only related with lambda
opened by dayu1979 2
An inconsistency of the input of IBLayer

In inference_demo.py, R = encoder.features[i]; while in the forward function of IIB, R = readout_feats[i]. Which should I use for training? And when I use R = readout_feats[i], Info is really large (5668875) at the beginning of training.

opened by lyx0208 1
$FileNotFoundError: [Errno 2] No such file or directory: './checkpoints_512/w_kernel_smooth\\ckpt_ks_G.pth'$

FileNotFoundError: [Errno 2] No such file or directory: './checkpoints_512/w_kernel_smooth\\ckpt_ks_G.pth'

Hello, I have the following problem now. Could you please provide me with this checkpoint? Thank you

FileNotFoundError:No such file or directory: './checkpoints_512/w_kernel_smooth\ckpt_ks_G.pth'

opened by an-zi-li 1
About the encoders, decoders and AII generators

According to the pseudo code "Algorithm 1" in the supp.pdf file, there are 2 pretrained encoders, 3 decoders and 2 AII generators.

For the encoders and decoders, if I only use 1 module and use it several times during the training, I have this error: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation It's OK to copy a encoder as it's pretrained and I do not need to optimize it during training. But for the decoder, I think its not proper to copy it 3 times, as according to my understanding of the algorithm, the three parts should use decoder with the same parameter.

For the AII generator, I think I just need to have two generators with the same structure, just as the cycle gan has two generators. And I can use the Lcyc in line 41 to optimize these two generators together. Is this understanding right?

opened by lyx0208 1
source-to-target pairs

Hi, thanks for your great work!

For the quantitative results in your work，I have questions about the correspondence of pair frames from source and target videos respectively.

(1) Did you randomly select 10 frames from each video or get the same pairs as FaceShifter?

(2) Could you provide the source-to-target pairs numbers for further fair comparison?

Thanks!

opened by ZekuiQin 1

Owner

Grace Hešeri

I'm currently working in the National Laboratory of Pattern Recognition (NLPR). My scientific interests are in the field of computer vision and machine learning

GitHub

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Face Identity Disentanglement via Latent Space Mapping - Implement in pytorch with StyleGAN 2 Description Pytorch implementation of the paper Face Ide

58 Dec 24, 2022

Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

37 Dec 3, 2022

Face Identity Disentanglement via Latent Space Mapping [SIGGRAPH ASIA 2020]

Face Identity Disentanglement via Latent Space Mapping Description Official Implementation of the paper Face Identity Disentanglement via Latent Space

150 Dec 7, 2022

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021) The implementation of Reducing Infromation Bottleneck for W

81 Dec 16, 2022

Official Implementation of Swapping Autoencoder for Deep Image Manipulation (NeurIPS 2020)

Swapping Autoencoder for Deep Image Manipulation Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, Richard Zhang UC

449 Dec 27, 2022

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

364 Dec 14, 2022

Official pytorch implementation of "Scaling-up Disentanglement for Image Translation", ICCV 2021.

41 Nov 29, 2022

Pytorch implementation of paper "Learning Co-segmentation by Segment Swapping for Retrieval and Discovery"

SegSwap Pytorch implementation of paper "Learning Co-segmentation by Segment Swapping for Retrieval and Discovery" [PDF] [Project page] If our project

41 Dec 10, 2022

Pytorch Implementation of Continual Learning With Filter Atom Swapping (ICLR'22 Spolight) Paper

Continual Learning With Filter Atom Swapping Pytorch Implementation of Continual Learning With Filter Atom Swapping (ICLR'22 Spolight) Paper If find t

11 Aug 29, 2022

Implementation of Bottleneck Transformer in Pytorch

Bottleneck Transformer - Pytorch Implementation of Bottleneck Transformer, SotA visual recognition model with convolution + attention that outperforms

621 Jan 6, 2023

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

34 Nov 16, 2022

Swapping face using Face Mesh with TensorFlow Lite

17 Apr 26, 2022

Rethinking of Pedestrian Attribute Recognition: A Reliable Evaluation under Zero-Shot Pedestrian Identity Setting

Pytorch Pedestrian Attribute Recognition: A strong PyTorch baseline of pedestrian attribute recognition and multi-label classification.

79 Dec 18, 2022

Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet

Reproduce ResNet-v2 using MXNet Requirements Install MXNet on a machine with CUDA GPU, and it's better also installed with cuDNN v5 Please fix the ran

531 Dec 4, 2022

Complete system for facial identity system. Include one-shot model, database operation, features visualization, monitoring

2 Dec 28, 2021

Official PyTorch Implementation for InfoSwap: Information Bottleneck Disentanglement for Identity Swapping

Related tags

Overview

InfoSwap: Information Bottleneck Disentanglement for Identity Swapping

Code usage

Paper

Results Across Large Gaps:

Results of 1024x1024 Pixels:

Results in Film Scenes:

Citation

Comments

太棒的思想了，看源码有个问题

An inconsistency of the input of IBLayer

FileNotFoundError: [Errno 2] No such file or directory: './checkpoints_512/w_kernel_smooth\\ckpt_ks_G.pth'

About the encoders, decoders and AII generators

source-to-target pairs

Owner

Grace Hešeri

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

Face Identity Disentanglement via Latent Space Mapping [SIGGRAPH ASIA 2020]

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Official Implementation of Swapping Autoencoder for Deep Image Manipulation (NeurIPS 2020)

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

Official pytorch implementation of "Scaling-up Disentanglement for Image Translation", ICCV 2021.

Pytorch implementation of paper "Learning Co-segmentation by Segment Swapping for Retrieval and Discovery"

Pytorch Implementation of Continual Learning With Filter Atom Swapping (ICLR'22 Spolight) Paper

Implementation of Bottleneck Transformer in Pytorch

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Swapping face using Face Mesh with TensorFlow Lite

Rethinking of Pedestrian Attribute Recognition: A Reliable Evaluation under Zero-Shot Pedestrian Identity Setting

Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet

Complete system for facial identity system. Include one-shot model, database operation, features visualization, monitoring

Complete system for facial identity system

Bottleneck Transformers for Visual Recognition

codes for Image Inpainting with External-internal Learning and Monochromic Bottleneck

Code for the paper: "On the Bottleneck of Graph Neural Networks and Its Practical Implications"