Official implementation for paper: Feature-Style Encoder for Style-Based GAN Inversion

InterDigital

Last update: Jan 3, 2023

Related tags

Deep Learning FeatureStyleEncoder

Overview

Feature-Style Encoder for Style-Based GAN Inversion

Official implementation for paper: Feature-Style Encoder for Style-Based GAN Inversion.

Code will be released soon.

Abstract We propose a novel architecture for GAN inversion, which we call Feature-Style encoder. The style encoder is key for the manipulation of the obtained latent codes, while the feature encoder is crucial for optimal image reconstruction. Our model achieves accurate inversion of real images from the latent space of a pre-trained style-based GAN model, obtaining better perceptual quality and lower reconstruction error than existing methods. Thanks to its encoder structure, the model allows fast and accurate image editing. Additionally, we demonstrate that the proposed encoder is especially well-suited for inversion and editing on videos. We conduct extensive experiments for several style-based generators pre-trained on different data domains. Our proposed method yields state-of-the-art results for style-based GAN inversion, significantly outperforming competing approaches.

Comments

What is the exact sturcture of feature prediction branch network?

Hi, sorry for bothering again.

I wonder if it is convenient for you to share the structure of the network? Specifically, the feature prediction branch and latent prediction branch. Thanks.

opened by LiUzHiAn 6
Pixel-wise reconstruction loss

Hi, thanks for contributing this excellent work.

Could you please tell whether the L2 recon. loss is applied on \hat{x}_2 = G(w, F) for the real image case? I feel a bit confused about this point when reading the paper. Thanks in advance.

opened by LiUzHiAn 4
The adapted pixel2style2pixel code is missing

Hi Xu-Yao, I've tried running the code and found the code under the pixel2style2pixel/ folder is missing I've tried using the code from the original implementation instead, but there are at least several changes like Generator's forward method getting input features and possibly many additional changes.

I'm not sure if you are still in the process of updating the repo, but thought I'd remind you just in case you simply forgot to add your adapted pixel2style2pixel/ code.

Looking forward to testing your encoder. looks very promising

opened by SelfishGene 1
Boundaries not working

Since there are no boundaries provided in the repo, I've tried my own to change facial attributes. Unfortunately, my boundaries work well if I use them only with pSp or e4e, but most of them do not work well with your encoder. They produce very ugly output images. I wonder why, since you also use StyleGAN2. So what is the reason and the sollution? And how do you look for directions using InterFaceGAN if its originally used for StyleGAN1?

opened by julkaztwittera 3

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

191 Dec 31, 2022

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

18 Dec 23, 2022

Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Likelihood-Regret Official implementation of Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020. T

33 Oct 12, 2022

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Alias-Free-Torch Simple torch module implementation of Alias-Free GAN. This repository including Alias-Free GAN style lowpass sinc filter @filter.py A

64 Dec 22, 2022

Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

RecycleD Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN

23 Nov 5, 2022

Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis

Chunked Autoregressive GAN (CARGAN) Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis [paper] [compan

150 Dec 6, 2022

Official Implementation of LARGE: Latent-Based Regression through GAN Semantics

LARGE: Latent-Based Regression through GAN Semantics [Project Website] [Google Colab] [Paper] LARGE: Latent-Based Regression through GAN Semantics Yot

83 Dec 6, 2022

This is the official Pytorch implementation of the paper "Diverse Motion Stylization for Multiple Style Domains via Spatial-Temporal Graph-Based Generative Model"

Diverse Motion Stylization (Official) This is the official Pytorch implementation of this paper. Diverse Motion Stylization for Multiple Style Domains

28 Dec 16, 2022

PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

VAENAR-TTS - PyTorch Implementation PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

67 Nov 14, 2022

Official implementation for paper: Feature-Style Encoder for Style-Based GAN Inversion

Related tags

Overview

Feature-Style Encoder for Style-Based GAN Inversion

You might also like...

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis

Official Implementation of LARGE: Latent-Based Regression through GAN Semantics

This is the official Pytorch implementation of the paper "Diverse Motion Stylization for Multiple Style Domains via Spatial-Temporal Graph-Based Generative Model"

PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

Comments

What is the exact sturcture of feature prediction branch network?

Pixel-wise reconstruction loss

The adapted pixel2style2pixel code is missing

Boundaries not working

Owner

InterDigital

Implementation for HFGI: High-Fidelity GAN Inversion for Image Attribute Editing

A collection of resources on GAN Inversion.

[CVPR 2021] Unsupervised 3D Shape Completion through GAN Inversion

A Simplied Framework of GAN Inversion

[IJCAI-2021] A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation"

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Official Implementation for "ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement" https://arxiv.org/abs/2104.02699

Transfer style api - An API to use with Tranfer Style App, where you can use two image and transfer the style

Official PyTorch implementation of Retrieve in Style: Unsupervised Facial Feature Transfer and Retrieval.

Chunkmogrify: Real image inversion via Segments