Hi, I'm interested in your RNN-GANs model but I'm still new to this domain. I'm trying to run your code but I get some problems. The first one is when I run the generating command: python generate.py --CKPT_PATH=/logs/Generator_RNN_CL_VL_TH-Discriminator_RNN-50-10-512-512-1509799522.05-/checkpoint --DISC_GRU_LAYERS=2 --GEN_GRU_LAYERS=2 The system outputs this error:

File "generate.py", line 20, in <module>
    _, inference_op = Generator(BATCH_SIZE, charmap_len, seq_len=SEQ_LEN)
  File "/.../rnn.wgan/model.py", line 51, in Generator_RNN_CL_VL_TH
    cells.append(rnn_cell(num_neurons))
TypeError: 'NoneType' object is not callable

The second one is when I train the model by running: python curriculum_training.py. It takes a lot of time, 3 days for training seq-1 to seq-4. I run this code on a server with 256CPU , 1536Core, Intel Xeon E5-4655v3 [6Core, 30M Cache, 2.9GHz], 32TB of RAM (DDR4).

Can you help me about that? Thank you in advance.

HI, I use the default setting for curriculum_training, than use --GENERATOR_MODEL=Generator_RNN_CL_VL_TH --DISCRIMINATOR_MODEL=Discriminator_RNN --ckpt_path=F:\NLP\rnn-wgan\logs\Generator_RNN_CL_VL_TH-Discriminator_RNN-50-10-512-512-1510731157.353556-\checkpoint for generate script parameters. While error happens in model.py. File "F:\NLP\rnn-wgan\model.py", line 52, in Generator_RNN_CL_VL_TH cells.append(rnn_cell(num_neurons)) TypeError: 'NoneType' object is not callable Could you plz give me some suggestions?

Hi, thanks for the repo, I think it's an original idea to apply GANs.

I would ask for some advice for adapting this code for lyrics. I have large availability of lyrics, but I don't have an idea to adapt the code for my task. Trying to be more clear: I would create a GAN that, at the end of the training steps, is able to generate lyrics (general format lyrics, not specified on particular artist) and at the same time use the trained discriminator (after training) to recognize lyrics that doesn't match format ( sort of spam detection).

Do you think it's a good idea? Could you suggest something in this way? Thanks

Simone

So I am researching in language models that can generate words.

Your results showed that RNN + GANs improves the quality of generated sequences compared to CNN+GANs. But do you think that the combination of RNN and GANs performs better than an independent RNN model?(LSTM, GRU) Because the resulting sentences you got from RNN+GANs are not coherent, and a well-trained RNN model can do the same job. Also, when I was training your model, I feel like using CL+VL+TH is very time-consuming. So is it really worth to train an RNN with GANs? Or the purpose of this project is just to prove that RNN could work well with GANs?

Thanks, Sida Sun

In the fisher GAN implementation, alpha_optimizer_op is not returned and is not run anywhere. Isn't it necessary to add the op as a control_dependency?

It seems current code can only work with fixed length text generation, then generator and discriminator are trained for fixed length input/output. I'm wondering is there a way to generate variable length text?

Could you share your experience of WGAN-GP vs. FisherGAN on learning language generator? Conceptually, I think 'sampling along the line between real sequence vs. generated sequence' of WGAN-GP sounds not natural for sequence because they can be different length. But I want to know which method works well in practice.

cf) If this is not appropriate topic for Issues, I will delete this post.

This PR consists of the following changes:

1. Fisher GAN
2. Recurrent highway network with multiplicative integration and normalizing tanh
3. Two Time Scale ADAM (this has been set as default)

I have kept as much naming conventions the same as much as possible. One big change made is that all the gru names are replaced with rnn as it allows us to swap different type of rnn cells much more easily assuming they have a state size of n.

On my text editor (sublime), there were some weird spacing issues when I first pulled your repo. Hopefully this doesn't conflict with your editor.

In the meantime, I'm considering cramer gan and have made a corresponding branch on my repo.

Includes fgan, recurrent highway rnn, and multiplicative integration.

Fgan seems to converge faster, but experience more mode collapse which drives me to pursue cramer gan

@amirbar thanks for releasing this code. I've been doing some experimentation, and I have found the following results:

1. Fisher GAN seems to keep approximately the same performance, but speeds up computation. I personally think it helps the discriminator more because the gradients don't vanish as much compared to WGAN-GP.

2. Simply increasing the GRU's state size to 1024 seems to improve performance as well.

3. As confirmed by @ofirpress on reddit, I can't get any temporal convolution discriminator's to work well.

I don't know if you're team is still pursuing this direction of research but just wanted to notify you in case you were.