Hi, I see that in your paper, the discriminator is updated with

I think the D appeared here means a "discriminator with sigmoid", But in the implementation, we always use "tf.nn.sigmoid_cross_entropy_with_logits" instead of explicit calculation of the sigmoid function.

In your paper, I think the gradient penalty is based on "discriminator with sigmoid" But in your codes, gradients = tf.gradients(discriminator(interpolates), [interpolates])[0] this is the gradient term w.r.t "discriminator without sigmoid"

In your paper (page 6. equation 1), you calculate the gradient penalty in this way:

So it looks like you simply add pixelwise Gaussian Noise with mean 0 and standard-deviation c to the inputs of the discriminator. Furthermore you suggest to use c = 10

However, in your code, you you calculate the perturbed inputs in this way:

def get_perturbed_batch(minibatch):
    return minibatch + 0.5 * minibatch.std() * np.random.random(minibatch.shape)

So here you add pixelwise uniform noise from the interval [0.0, 0.5 * minibatch.std() ), which is very different to the formulation of noise in the paper. Furthermore your code still contains the interpolation issue from https://github.com/kodalinaveen3/DRAGAN/issues/5

Of course you wrote "We use small pixel-level noise but it is possible to find better ways of imposing this penalty. However, this exploration is beyond the scope of our paper." But maybe there are important reasons why you use this type of X_p generation in the code?

interpolates = X + (alpha*differences)
= X + U[0, 1] (X_p - X)
= X + U[0, 1](X + 0.5*std * U[0,1] - X)
= X + U[0, 1] * 0.5 std * U[0, 1]
= X + 0.5 * std * U[0, 1]^2
~= X_p

so it appears that X_p can be directly used instead of interpolates.

And the paper actually used X_p instead of interpolates.

Hi, Here the sum of squares is computed slopes = tf.sqrt(tf.reduce_sum(tf.square(gradients), reduction_indices=[1]))

However, when the input image is extremely large, the dimension of gradients would be huge. It seems possible that the resulting slopes would be extremely large compared to 1. Is this the case?

When I try to apply this loss function to my model, modified from DCGAN, I got NAN loss, I wonder will this modified vanilla function will lead to this issue? Thanks!

I'm not sure if this is abandoned or not, but I spent a considerable amount of time trying to replicate the notebook without looking at the closed issues.

Teaches me a lesson not to look at the closed issues!

See (which have been answered by the authors) https://github.com/kodalinaveen3/DRAGAN/issues/6 https://github.com/kodalinaveen3/DRAGAN/issues/5

It also looks like several implementations have implemented based on the old version of the code, e.g. https://github.com/opendp/smartnoise-sdk/blob/main/synth/snsynth/pytorch/nn/patectgan.py

One argument of tf.nn.sigmoid_cross_entropy_with_logits() seems changed from "targets" to "labels" in recent released version of Tensorflow. Refer to https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits

Hi his DRAGAN Code for Tensorflow

In his code, you said that alpha should have a value between -1 and 1, right? The alpha between 0 and 1 is bug ... ??????? However, Most of the code seems to write alpha between 0 and 1.

What is right?