Tensorflow Implementation of the paper "Spectral Normalization for Generative Adversarial Networks" (ICML 2017 workshop)

Overview

tf-SNDCGAN

Tensorflow implementation of the paper "Spectral Normalization for Generative Adversarial Networks" (https://www.researchgate.net/publication/318572189_Spectral_Normalization_for_Generative_Adversarial_Networks, ICML 2017)

The implementation is based on the author's original code at: https://github.com/pfnet-research/chainer-gan-lib

This implementation works for tensorflow default data format "NHWC"

Spectral Normalization for Generative Adversarial Networks:

This method enforces Lipschitz-1 condition on the Discrminator of Wasserstein-GAN by normalizing its weight matrices with their own respective maximum singular value. This can be used together with Gradient Penalty in the paper "Improved Training of Wasserstein GAN".

The author uses a fast approximation method to compute the maximum singular value of weight matrices.

Quick run:

Keras is required for loading Cifar10 data set

python3 train.py

How to use spectral normalization:

# Import spectral norm wrapper
from libs.sn import spectral_normed_weight
# Create weight variable
W = tf.Variable(np.random.normal(size=[784, 10], scale=0.02), name='W', dtype=tf.float32)
# name of tf collection used for storing the update ops (u)
SPECTRAL_NORM_UPDATE_OPS = "spectral_norm_update_ops"
# call wrapping function, W_bar will be the spectral normed weight matrix
W_bar = spectral_normed_weight(W, num_iters=1, update_collection=SPECTRAL_NORM_UPDATE_OPS)
# Get the update ops
spectral_norm_update_ops = tf.get_collection(SPECTRAL_NORM_UPDATE_OPS)
...
# During training, run the update ops at the end of the iteration
for iter in range(max_iters):
    # Training goes here
    ...
    # Update ops at the end
    for update_op in spectral_norm_update_ops:
        sess.run(update_op)

For an example, see the file test_sn_implementation.py

Training curve:

Generated image samples on Cifar10:

Inception score:

After using in place batch norm update and use the optimal training parameters from the paper, I was able to match their claimed Inception score at 100k iteration: 7.4055686 +/- 0.087728456

The official github repostiory has an inception score of 7.41

Issues:

  • GPU under-utilization: The original implementation of the author in chainer uses 80%+ GPU most of the time. On an NVIDIA GTX 1080TI, their implementation run at nearly 3 iterations/s. This implementation use less than 50% GPU and run at less than 2 iterations/s. Solved. It was the global_step assignment that makes tensorflow create new assign node for graph each iteration, slow down the execution. This also made the graph become very large over time leading to gigantic event files. GPU utilization is now around 85+%

  • No Fréchet Inception Distance (https://arxiv.org/abs/1706.08500) evaluation yet.

Comments
  • How do I generate a lot of images?

    How do I generate a lot of images?

    How do I generate a lot of images?

    python train.py
    

    if max_iter = 100000, then only 64 images are generated

    I want to generate a lot of features, but also a lot of images.

    how ?

    opened by taki0112 2
  • I want to apply to tf.layers.con2d

    I want to apply to tf.layers.con2d

    If I want to apply to tf.layers.con2d, what should I do? I want to put it in the kernel initializer.

    for example

        x = tf.layers.conv2d(inputs=x, filters=channels,
                                          kernel_size=kernel, 
                                          kernel_initializer=spectral_normed_weight(),
                                          strides=stride)
    

    Should I just use tf.nn.conv2d... ?

    opened by taki0112 1
  • Data format?

    Data format?

    opened by wchen342 1
  • Why do you use `update collection`?

    Why do you use `update collection`?

    While skimming through your code, I don't understand why do you need update_collection variable here, https://github.com/minhnhat93/tf-SNDCGAN/blob/master/train.py#L39

    If i don't add update collection in the code. just write like below

        with tf.control_dependencies([u.assign(u_hat)]):
            w_norm = tf.reshape(w_norm, w_shape)
    

    d_real and d_loss both use the above procedure. Is there anything wrong? thx

    opened by IPNUISTlegal 0
  • warning in `spectral_normed_weight`

    warning in `spectral_normed_weight`

    Hi! I've been using your implementation of SNGAN and I have some questions. If I follow the training procedure intrain.py, then the warning in the spectral_normed_weight function will print

    Setting update_collection to None will make u being updated every W execution. This maybe undesirable. Please consider using a update collection instead.

    I wonder if the warning was meaningful. I also found that the way you used the collection api is different in train.py and test_sn_implementation.py. Which way should I follow?

    Thanks!

    opened by unilight 0
  • Input Error for Evaluation

    Input Error for Evaluation

    There comes a error when doing evaluation: Cannot feed value of shape (100, 32, 32, 3) for Tensor 'ExpandDims:0' at libs/inception_score/model.py line 45.

    Thanks for sharing.

    opened by OswinGuai 1
Owner
Nhat M. Nguyen
Nhat M. Nguyen
PyTorch implementation of NIPS 2017 paper Dynamic Routing Between Capsules

Dynamic Routing Between Capsules - PyTorch implementation PyTorch implementation of NIPS 2017 paper Dynamic Routing Between Capsules from Sara Sabour,

Adam Bielski 475 Dec 24, 2022
Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Language Emergence in Multi Agent Dialog Code for the Paper Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog Satwik Kottur, José M.

Karan Desai 105 Nov 25, 2022
A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017

Semantic Image Synthesis via Adversarial Learning This is a PyTorch implementation of the paper Semantic Image Synthesis via Adversarial Learning. Req

Seonghyeon Nam 146 Nov 25, 2022
Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

GANs for Biological Image Synthesis This codes implements the ICCV-2017 paper "GANs for Biological Image Synthesis". The paper and its supplementary m

Anton Osokin 95 Nov 25, 2022
Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Language Emergence in Multi Agent Dialog Code for the Paper Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog Satwik Kottur, José M.

Karan Desai 105 Nov 25, 2022
The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.

Face Alignment in Full Pose Range: A 3D Total Solution By Jianzhu Guo. [Updates] 2020.8.30: The pre-trained model and code of ECCV-20 are made public

Jianzhu Guo 3.4k Jan 2, 2023
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
Code for the USENIX 2017 paper: kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels

kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels Blazing fast x86-64 VM kernel fuzzing framework with performant VM reloads for Linux, MacOS an

Chair for Sys­tems Se­cu­ri­ty 541 Nov 27, 2022
The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

DS3L This is the code for paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020. Setups The code is implem

Guolz 36 Oct 19, 2022
An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge.

Bottom-Up and Top-Down Attention for Visual Question Answering An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge. The

Hengyuan Hu 731 Jan 3, 2023
Code for the ICML 2021 paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

ViLT Code for the paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision" Install pip install -r requirements.txt pip

Wonjae Kim 922 Jan 1, 2023
Code for the ICML 2021 paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

ViLT Code for the paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision" Install pip install -r requirements.txt pip

Wonjae Kim 922 Jan 1, 2023
Code for ICML 2021 paper: How could Neural Networks understand Programs?

OSCAR This repository contains the source code of our ICML 2021 paper How could Neural Networks understand Programs?. Environment Run following comman

Dinglan Peng 115 Dec 17, 2022
An interpreter for RASP as described in the ICML 2021 paper "Thinking Like Transformers"

RASP Setup Mac or Linux Run ./setup.sh . It will create a python3 virtual environment and install the dependencies for RASP. It will also try to insta

null 141 Jan 3, 2023
Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng Internati

Princeton Vision & Learning Lab 115 Jan 4, 2023
Code for Fold2Seq paper from ICML 2021

[ICML2021] Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design Environment file: environment.yml Data and Feat

International Business Machines 43 Dec 4, 2022
Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation", Haoxiang Wang, Han Zhao, Bo Li.

Bridging Multi-Task Learning and Meta-Learning Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Trainin

AI Secure 57 Dec 15, 2022
Code for the paper "Adversarially Regularized Autoencoders (ICML 2018)" by Zhao, Kim, Zhang, Rush and LeCun

ARAE Code for the paper "Adversarially Regularized Autoencoders (ICML 2018)" by Zhao, Kim, Zhang, Rush and LeCun https://arxiv.org/abs/1706.04223 Disc

Junbo (Jake) Zhao 399 Jan 2, 2023