DeLiGAN - This project is an implementation of the Generative Adversarial Network

Last update: Sep 13, 2022

Overview

DeLiGAN

This project is an implementation of the Generative Adversarial Network proposed in our CVPR 2017 paper - DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data. Via this project, we make two contributions:

We propose a simple but effective modification of the GAN framework for settings where training data is diverse yet small in size.
We propose a modification of inception-score proposed by Salimans et al. Our modified inception-score provides a single, unified measure of inter-class and intra-class variety in samples generated by a GAN.

Dependencies

The code for DeLiGAN is provided in Tensorflow 0.10 for the MNIST and Toy dataset, and in Theano 0.8.2 + Lasagne 0.2 for the CIFAR-10 and Sketches dataset. This code was tested on a Ubuntu 14.04 workstation hosting a NVIDIA Titan X GPU.

Datasets

This repository includes implementations for 4 different datasets.

Toy (self generated unimodal and bimodal gaussians)
MNIST (http://www.cs.toronto.edu/~gdahl/mnist.npz.gz)
CIFAR-10 (https://www.cs.toronto.edu/~kriz/cifar.html)
Sketches (http://cybertron.cg.tu-berlin.de/eitz/projects/classifysketch/)

The models for evaluating DeLiGAN on these datasets can be found in our repo. The details for how to download and lay out the datasets can be found in src/datasets/README.md

Usage

Training DeLiGAN models

To run any of the models

First download the datasets and store them in the respective sub-folder of the datasets folder (src/datasets/)
To run the model on any of the datasets, go to the respective src folders and run the dg_'dataset'.py file in the respective dataset folders with two arguments namely, --data_dir and --results_dir. For example, starting from the top-level folder,

cd src/sketches 
python dg_sketches.py --data_dir ../datasets/sketches/ --results_dir ../results/sketches

Note that the results_dir needs to have 'train' as a sub-folder.

Modified inception score

For example, to obtain the modified inception scores on CIFAR

Download the inception-v3 model (http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz.) and store it in src/modified_inception_scores/cifar10/
Generate samples using the model trained in the dg_cifar.py and copy it to src/modified_inception_scores/cifar10/
Run transfer_cifar10_softmax_b1.py to transfer learn the last layer.
Perform the modifications detailed in the comments in transfer_cifar10_softmax_b1.py and re-run it to evaluate the inception scores.
The provided code can be modified slightly to work for sketches as well by following the comments provided in transfer_cifar10_softmax_b1.py

Parts of the code in this implementation have been borrowed from the Improved-GAN implementation by OpenAI (T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2226–2234, 2016.)

Cite

@inproceedings{DeLiGAN17,
  author = {Gurumurthy, Swaminathan and Sarvadevabhatla, Ravi Kiran and R. Venkatesh Babu},
  title = {DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data},
  booktitle = {Proceedings of the 2017 Conference on Computer Vision and Pattern Recognition},
  location = {Honolulu, Hawaii, USA}
 }

Q&A

Please send message to [email protected] if you have any query regarding the code.

Comments

fixed batch size

if I want to apply deli-GAN's spirits to some other application like conditional GAN , so the batch size have to fixed? I think this is not convenient.

opened by SchafferZhang 3
Mode Collapse for toy dataset?

Hi,

I was running your script in toy directory to reproduce results shown in paper, however ending up with mode collapse for both unimodal and multimodal data.

To deduce the reflection on initialization, I've run this test for 5 times on both datasets, results are pretty similar.

Any comments?

opened by hangg7 2
generating same sample

Hi, when I utilize the trained model to generate the samples, which the corresponding code is

#Uncomment this block for generating GAN samples from given model ''' f = np.load(args.results_dir + '/disc_params1180.npz') param_values = [f['arr_%d' % i] for i in range(len(f.files))] for i,p in enumerate(disc_params): p.set_value(param_values[i]) print("disc_params fed") f = np.load(args.results_dir + '/gen_params1180.npz') param_values = [f['arr_%d' % i] for i in range(len(f.files))] for i,p in enumerate(gen_params): p.set_value(param_values[i]) print("gen_params fed") samples=[] for i in range(50): sample_x = samplefun() samples.append(sample_x) samples = np.concatenate(samples,0) print(samples) #sys.exit() np.save(args.results_dir + '/samples50k.npy', samples) print("samples saved") sys.exit() '''

but for each iteration i, the image in the sample_x are diverse, but every sample_x is the same. what' wrong with it?

opened by SchafferZhang 1
Some questions about the project

Two questions: 1.Do you find performance reduction if you increase the size of the generated image? Such as the dataset of the sketch? Can you give some suggestions if I want to increase the size of the results? 2.How to modify the network if I want to generate specific kind of images, just like the conditional GAN? Any suggestions? Thanks!

opened by OwalnutO 1
Latent Space

Hi, I noticed the paper said the trained latent space is a mixed Gaussian distribution with trainable variance and expectation:

In particular, we propose a reparameterization of the latent space as a Mixture- of-Gaussians model.

However, it seems that in the script the latent space applied here is a uniform distribution with trainable variance and expectation: display_z = np.random.uniform(-1.0, 1.0, [batchsize, z_dim]).astype(np.float32) I don't quite understand this inconsistency.

opened by Bob-RUC 5

DeLiGAN - This project is an implementation of the Generative Adversarial Network

Related tags

Overview

DeLiGAN

Dependencies

Datasets

Usage

Training DeLiGAN models

Modified inception score

Cite

Q&A

Comments

fixed batch size

Mode Collapse for toy dataset?

generating same sample

Some questions about the project

Latent Space

Owner

Video Analytics Lab -- IISc

House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent for Professional Architects

Python code to generate art with Generative Adversarial Network

Unofficial implement with paper SpeakerGAN: Speaker identification with conditional generative adversarial network

A method that utilized Generative Adversarial Network (GAN) to interpret the black-box deep image classifier models by PyTorch.

Collection of TensorFlow2 implementations of Generative Adversarial Network varieties presented in research papers.

Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Pytorch Implementation of Adversarial Deep Network Embedding for Cross-Network Node Classification

Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Implementation for Paper "Inverting Generative Adversarial Renderer for Face Reconstruction"

Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) in PyTorch

Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations

Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks"

Official implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks"

A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

Alias-Free Generative Adversarial Networks (StyleGAN3) Official PyTorch implementation

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

Implementation based on Paper - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling