[NeurIPS 2021] Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

Liming Jiang

Last update: Nov 25, 2022

Related tags

Deep Learning generative-adversarial-network gan generative-model image-generation image-synthesis few-shot limited-data stylegan2 neurips-2021

Overview

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data (NeurIPS 2021)

This repository will provide the official PyTorch implementation for the following paper:

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
Liming Jiang, Bo Dai, Wayne Wu and Chen Change Loy
In NeurIPS 2021.
Project Page | Paper | Poster | Slides | YouTube Demo

Abstract: Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting, the underlying cause that impedes the generator's convergence. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator. As an alternative method to existing approaches that rely on standard data augmentations or model regularization, APA alleviates overfitting by employing the generator itself to augment the real data distribution with generated images, which deceives the discriminator adaptively. Extensive experiments demonstrate the effectiveness of APA in improving synthesis quality in the low-data regime. We provide a theoretical analysis to examine the convergence and rationality of our new training strategy. APA is simple and effective. It can be added seamlessly to powerful contemporary GANs, such as StyleGAN2, with negligible computational cost.

convergence_demo.mp4

Updates

[09/2021] The paper of APA is accepted by NeurIPS 2021.

Code Release Date

Our code will be made publicly available by the end of this month (November 2021). Please stay tuned.

Results

Effectiveness on Various Datasets

Effectiveness Given Different Data Amounts

Overfitting and Convergence Analysis

Comparison with Other State-of-the-Art Solutions

Higher-Resolution Examples (1024 × 1024) on FFHQ-5k (~7% data)

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{jiang2021DeceiveD,
  title={{Deceive D: Adaptive Pseudo Augmentation} for {GAN} Training with Limited Data},
  author={Jiang, Liming and Dai, Bo and Wu, Wayne and Loy, Chen Change},
  booktitle={NeurIPS},
  year={2021}
}

License

Comments

What's the difference between images 1024x1024 and tfrecords on the FFHQ download page?

What's the difference between images 1024x1024 and tfrecords on the FFHQ download page? The tfrecords have 273 GB, and the images 1024 x 1024 have 89.2 GB. If I use Pytorch, can I directly download images 1024x1024 and use dataset_tool.py to create FFHQ 256x256?

opened by Yongssss 4
Increase resolution of images.

Hello, First of all, congrats for your nice work. I have trained DeceiveD/APA with my set which are formed by 128 x 128 images. I am wondering whether I can generate higher resolution images (1024 x 1024) with your GAN based on low resolution ones. When preparing the data, I ran something like:

python dataset_tool.py --source=/path/to/image/folder --dest=/path/to/archive.zip --width=128 --height=128 --max-images=100

To train (stylegan2 for 1024 x 1024):

python train.py --outdir=./experiments --gpus=4 --data=/path/to/mydataset.zip --mirror=1 --cfg=stylegan2 --aug=apa --with-dataaug=true

Inference:

python generate.py --outdir=out --trunc=1 --seeds=1000-1199 --network=/path/to/checkpoint/pkl

But after the inference, the output were 128 x 128 images (as the original ones).

Thus, can we use it for creating higher resolution images?

Thank you.

opened by vsantjr 2
enviorment

My setting is : ubuntu20.04 ; python 3.7.0; pytorch1.7.1+cu110; cuda11.1. The code can run, but there is an issue: [ERROR] Sorry, cuMemHostGetDevicePointer_v2() is not implemented in the current version. If user application keeps reporting errors, please use Ctrl+C to exit. how should I solve this issue?

opened by zyqm123 2
Add Anaconda environment.yml

Firstly: Awesome project!

You recommend using Anaconda in the install description. Would it be possible to add an environment.yml with all needed dependencies to the repository to ease the installation process?

opened by alternative-imaginaries 1
Add Docker environment & web demo

Hey @EndlessSora ! 👋

This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

This also means we can make a web page where other people can try out your model! View it here: https://replicate.ai/endlesssora/deceived

Do claim your page here so you can own the page, customise the Example gallery as you like, push any future update to the web demo, and we'll feature it on our website and tweet about it too.

In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊

opened by chenxwh 0

[NeurIPS 2021] Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

Related tags

Overview

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data (NeurIPS 2021)

Updates

Code Release Date

Results

Effectiveness on Various Datasets

Effectiveness Given Different Data Amounts

Overfitting and Convergence Analysis

Comparison with Other State-of-the-Art Solutions

Higher-Resolution Examples (1024 × 1024) on FFHQ-5k (~7% data)

Citation

License

Comments

What's the difference between images 1024x1024 and tfrecords on the FFHQ download page?

Increase resolution of images.

enviorment

Add Anaconda environment.yml

Add Docker environment & web demo

Owner

Liming Jiang

An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

Image transformations designed for Scene Text Recognition (STR) data augmentation. Published at ICCV 2021 Workshop on Interactive Labeling and Data Augmentation for Vision.

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)

The source codes for ACL 2021 paper 'BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data'

The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Online Pseudo Label Generation by Hierarchical Cluster Dynamics for Adaptive Person Re-identification

Collapse by Conditioning: Training Class-conditional GANs with Limited Data

This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels].

[WWW 2021] Source code for "Graph Contrastive Learning with Adaptive Augmentation"

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly

Regularizing Generative Adversarial Networks under Limited Data (CVPR 2021)

Experiments on Flood Segmentation on Sentinel-1 SAR Imagery with Cyclical Pseudo Labeling and Noisy Student Training

StyleGAN2 with adaptive discriminator augmentation (ADA) - Official TensorFlow implementation

Frequency Spectrum Augmentation Consistency for Domain Adaptive Object Detection

This is an official PyTorch implementation of Task-Adaptive Neural Network Search with Meta-Contrastive Learning (NeurIPS 2021, Spotlight).

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

Code and data for ACL2021 paper Cross-Lingual Abstractive Summarization with Limited Parallel Resources.