Implement a working version of gan based on the original 11_gan.py.

Main modifications are as follows:

• G output range from 0 to 255.
• Activation function changes from relu to tanh.
• G objective and D objective change.
• training two step of G and one step of D in one batch.

Hi,

Thanks for those examples, they are very useful.

If you are still working on more examples, it'd be awesome to have an example of time series forecasting using LSTM. There's a lot of search for this, and no working example so far.

It's mostly the preparation of the input data that is confusing for me.

Fixed tensorflow 0.9 API compatibility

1. replaced "from tensorflow.models.rnn import rnn, rnn_cell" with "tf.nn.rnn_cell"
2. added state_is_tuple=True (https://www.tensorflow.org/versions/r0.9/api_docs/python/nn.html#recurrent-neural-networks)
3. using dtype=tf.float32 instead of init_state

1. Renamed files so that they are displayed in the correct order on Github: 1_foo.py -> 01_foo.py
2. Topic items in README.md link to *.py files directly

I think we need to update lstm:

 File "07_lstm.py", line 3, in <module>
    from tensorflow.models.rnn import rnn, rnn_cell
  File "/home/travis/virtualenv/python2.7.10/lib/python2.7/site-packages/tensorflow/models/rnn/rnn.py", line 21, in <module>
    raise ImportError("This module is deprecated.  Use tf.nn.rnn_* instead.")
ImportError: This module is deprecated.  Use tf.nn.rnn_* instead.

I tried to run the ipynb file on my Mac and got this error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/anaconda/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py in apply_op(self, op_type_name, name, **keywords)
    489                 as_ref=input_arg.is_ref,
--> 490                 preferred_dtype=default_dtype)
    491           except ValueError:

~/anaconda/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype)
    656         if ret is None:
--> 657           ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
    658 

~/anaconda/lib/python3.5/site-packages/tensorflow/python/framework/ops.py in _TensorTensorConversionFunction(t, dtype, name, as_ref)
    570         "Tensor conversion requested dtype %s for Tensor with dtype %s: %r"
--> 571         % (dtype.name, t.dtype.name, str(t)))
    572   return t

ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: 'Tensor("nce_loss/Reshape_1:0", shape=(?, 1, ?), dtype=float32)'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-3-9796681a13df> in <module>()
     19 #   tf.nn.nce_loss(weights, biases, inputs, labels, num_sampled, num_classes ...)
     20 # It automatically draws negative samples when we evaluate the loss.
---> 21 loss = tf.reduce_mean(tf.nn.nce_loss(nce_weights, nce_biases, train_labels, embed, num_sampled, voc_size))
     22 # Use the adam optimizer
     23 train_op = tf.train.AdamOptimizer(1e-1).minimize(loss)

~/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn.py in nce_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true, sampled_values, remove_accidental_hits, partition_strategy, name)
   1189       remove_accidental_hits=remove_accidental_hits,
   1190       partition_strategy=partition_strategy,
-> 1191       name=name)
   1192   sampled_losses = sigmoid_cross_entropy_with_logits(
   1193       logits, labels, name="sampled_losses")

~/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/nn.py in _compute_sampled_logits(weights, biases, inputs, labels, num_sampled, num_classes, num_true, sampled_values, subtract_log_q, remove_accidental_hits, partition_strategy, name)
   1049     row_wise_dots = math_ops.mul(
   1050         array_ops.expand_dims(inputs, 1),
-> 1051         array_ops.reshape(true_w, new_true_w_shape))
   1052     # We want the row-wise dot plus biases which yields a
   1053     # [batch_size, num_true] tensor of true_logits.

~/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py in mul(x, y, name)
   1517     A `Tensor`. Has the same type as `x`.
   1518   """
-> 1519   result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
   1520   return result
   1521 

~/anaconda/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py in apply_op(self, op_type_name, name, **keywords)
    510                   "%s type %s of argument '%s'." %
    511                   (prefix, dtypes.as_dtype(attrs[input_arg.type_attr]).name,
--> 512                    inferred_from[input_arg.type_attr]))
    513 
    514           types = [values.dtype]

TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.

when trying to execute the second last line of the second last block loss = tf.reduce_mean(tf.nn.nce_loss(nce_weights, nce_biases, train_labels, embed, num_sampled, voc_size)) I double checked the document, but still cannot fix the issue. Is there any idea what is going wrong? Python: 3.5 tensorflow: 0.11.0 OS: Sierra 10.12.2

Many thanks

When I run 01_linear_regression.py in my pc, terminal says "WARNING:tensorflow:From 01_linear_regression.py:27 in .: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use tf.global_variables_initializer instead." So, I changed tf.initialize_all_variables().run() to tf.global_variables_initializer().run()

Previously, when the len(trX) is completely divisible by batch_size, the training skipped over the last batch because end=len(trX) would never occur. Corrected this and now the loop will go over the entire training set if batch_size can cleanly fit in.

Hi

I am new to tensorflow, and I am stuck at your logistic_expression example. I trained and saved the model, restored it, but I am unable to use it. Can you help me with that? I tried:

print (sess.run(predict_op, feed_dict={X: teX[66], Y: teY[66]})) I am expecting a 6 here as output (or a number at least :D) but I got an error:

ValueError: Cannot feed value of shape (784,) for Tensor 'Placeholder:0', which has shape '(?, 784)'

Hello! @nlintz , your TensorFlow tutorials are very good, and they are also compact, comparing to other Tutorials/Examples on github.

But I need some help with understanding of 4 modern net. Please help me understand these lines:

30 line: w_h = init_weights([784, 625]) So, 784 is 28x28, which is flatten image data array. But what is 625 ? We also have next line: w_h2 = init_weights([625, 625]) Is 625 something related to batches/hidden layer units ?

47 line: for i in range(100): What is 100 in that line? Is that epoches count ?

48 line: for start, end in zip(range(0, len(trX), 128), range(128, len(trX)+1, 128)): Cannot understand how that is working at all. Could you please explain what we are doing here? Is that 128 - batch size ? What range of mnist.train.images we are taking? All 55k ?

Thank you!

I hope to get an answer

Hi,

I noticed there was no GAN in the repo, and I was wondering whether you think it'd be useful to have one. I added the python script, and a notebook with more comments to explain some of the choices.

Please let me know your thoughts!

Hello! I saw a really good code. Do not use a mutable data structure for argument default. and code refectoring was performed. Changed code def G(z, w=g_weights): and def G(z, w=None): if w is None: w = g_weights Have a great day!

In 10_save_restore_net.ipynb file, you save the model at each epoch. When you interrupt it with keyboard and starting running again, the model is restored from the epoch where the interruption occurred and it computes the loss (predictions in your case) at that epoch once again. For example, Model runs until epoch 2 and then I interrupt it:

Start from: 0 0 0.9374 1 0.9634 2 0.9697

After I re-run it, ./ckpt_dir/model.ckpt-2 INFO:tensorflow:Restoring parameters from ./ckpt_dir/model.ckpt-2 Start from: 2 2 0.9742 3 0.9747 4 0.9774 5 0.9768 6 0.9783 7 0.9786 8 0.9785 .....

It is computing the loss (predictions in your case) for the epoch 2 once again.

How correct is that?

Can I do start = global_step.eval() + 1?

All in the title,

To solve it, replace line 69-70

G_opt = tf.train.AdamOptimizer().minimize(G_obj, var_list=g_weights.values()) D_opt = tf.train.AdamOptimizer().minimize(D_obj, var_list=d_weights.values())

by

G_opt = tf.train.AdamOptimizer().minimize(G_obj, var_list=list(g_weights.values())) D_opt = tf.train.AdamOptimizer().minimize(D_obj, var_list=list(d_weights.values()))

Maybe due to a TensorFlow update?

Best, Simon