A crash course in six episodes for software developers who want to become machine learning practitioners.

Can you please explain the code for periph_rois ,I mean what it is for? and I am not able to understand the significance of 'expand' variable in traineryolo/box_utils.py

Hello, I've just cloned the repo and trying out the first mnist_1.0_softmax example that attempts to download from https://storage.googleapis.com/cvdf-datasets/mnist/ currently not accessible... other sources?

After following the instructions on a Windows 10 machine without GPU support, I encountered the issue described here: https://github.com/tensorflow/tensorflow/issues/17393 I also found the fix at that same location. Also, when running mnist_1.0_softmax.py in the Anaconda shell, I had to use "python" instead of "python3".

utils_display.picture_this_4() does not run (without any changes).

I get the below error information running jupyter on Ubuntu desktop 18.04. It is the same problem with:

utils_display.picture_this_6()

By commenting out these the notebook runs without problems.

And thanks by the way for the great videos and playgrounds, they really upgrade ones neural network understanding and learning!

AttributeError Traceback (most recent call last) in () 7 visu_dates = evdates[START_DATE:END_DATE] 8 ----> 9 utils_display.picture_this_4(visu_temperatures, visu_dates)

~/git-public/tensorflow-without-a-phd/tensorflow-rnn-tutorial/tutorial/utils_display.py in picture_this_4(temperatures, dates) 73 if interpolated_sequence: 74 # light shade of red just for visibility ---> 75 plt.axvspan(startdate+np.timedelta64(-5, 'D'), stopdate+np.timedelta64(6, 'D'), facecolor='#FFCCCC', alpha=1) 76 # actual interpolated region 77 plt.axvspan(startdate+np.timedelta64(-1, 'D'), stopdate+np.timedelta64(1, 'D'), facecolor='#FF8888', alpha=1)

/usr/local/lib/python3.6/dist-packages/matplotlib/pyplot.py in axvspan(xmin, xmax, ymin, ymax, hold, **kwargs) 2753 mplDeprecation) 2754 try: -> 2755 ret = ax.axvspan(xmin, xmax, ymin=ymin, ymax=ymax, **kwargs) 2756 finally: 2757 ax._hold = washold

/usr/local/lib/python3.6/dist-packages/matplotlib/axes/_axes.py in axvspan(self, xmin, xmax, ymin, ymax, **kwargs) 904 905 # first we need to strip away the units --> 906 xmin, xmax = self.convert_xunits([xmin, xmax]) 907 ymin, ymax = self.convert_yunits([ymin, ymax]) 908

/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py in convert_xunits(self, x) 189 if ax is None or ax.xaxis is None: 190 return x --> 191 return ax.xaxis.convert_units(x) 192 193 def convert_yunits(self, y):

/usr/local/lib/python3.6/dist-packages/matplotlib/axis.py in convert_units(self, x) 1524 return x 1525 -> 1526 ret = self.converter.convert(x, self.units, self) 1527 return ret 1528

/usr/local/lib/python3.6/dist-packages/matplotlib/dates.py in convert(value, unit, axis) 1814 The unit and axis arguments are not used. 1815 """ -> 1816 return date2num(value) 1817 1818 @staticmethod

/usr/local/lib/python3.6/dist-packages/matplotlib/dates.py in date2num(d) 450 if not d.size: 451 return d --> 452 return _to_ordinalf_np_vectorized(d) 453 454

~/.local/lib/python3.6/site-packages/numpy/lib/function_base.py in call(self, *args, **kwargs) 2753 vargs.extend([kwargs[_n] for _n in names]) 2754 -> 2755 return self._vectorize_call(func=func, args=vargs) 2756 2757 def _get_ufunc_and_otypes(self, func, args):

~/.local/lib/python3.6/site-packages/numpy/lib/function_base.py in _vectorize_call(self, func, args) 2823 res = func() 2824 else: -> 2825 ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args) 2826 2827 # Convert args to object arrays first

~/.local/lib/python3.6/site-packages/numpy/lib/function_base.py in _get_ufunc_and_otypes(self, func, args) 2783 2784 inputs = [arg.flat[0] for arg in args] -> 2785 outputs = func(*inputs) 2786 2787 # Performance note: profiling indicates that -- for simple

/usr/local/lib/python3.6/dist-packages/matplotlib/dates.py in _to_ordinalf(dt) 253 tzi = UTC 254 --> 255 base = float(dt.toordinal()) 256 257 # If it's sufficiently datetime-like, it will have a date() method

AttributeError: 'numpy.datetime64' object has no attribute 'toordinal'

when I run mnist_1.0_softmax.py in annconda cmd with python 3.6 on win10, it reported errors, who can help me with it? many thanks. I type the followings: (base) F:\tensorflow-without-a-phd\tensorflow-mnist-tutorial>python mnist_1.0_softmax.py

then it reported: Traceback (most recent call last): File "mnist_1.0_softmax.py", line 17, in <module> import tensorflowvisu File "F:\tensorflow-without-a-phd\tensorflow-mnist-tutorial\tensorflowvisu.py", line 27, in <module> tf.set_random_seed(0) AttributeError: module 'tensorflow' has no attribute 'set_random_seed'

There was a compatibility issue from TensorFlow 1.8 to 1.9, where the variable reuse behavior was changed when building a model with Keras.

Related issue: https://github.com/GoogleCloudPlatform/tensorflow-without-a-phd/issues/20 (Possibly related issue: https://github.com/GoogleCloudPlatform/tensorflow-without-a-phd/issues/11)

The 'git clone' command refers to 'https://github.com/martin-gorner/tensorflow-mnist-tutorial.git' which is the old MNIST repository (and contains only README).

After the end of the 1st epoch:

>>>>>>> epoch 1
>>> Rollout phase
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/pawel/M/outrun/tensorflow-without-a-phd/tensorflow-rl-pong/trainer/task.py", line 292, in <module>
    main(args)
  File "/home/pawel/M/outrun/tensorflow-without-a-phd/tensorflow-rl-pong/trainer/task.py", line 204, in main
    _ = raw_input('episode done, press Enter to replay')
NameError: name 'raw_input' is not defined

raw_input() is not supported in Python 3.

As per Guido's explanation: https://docs.python.org/3/whatsnew/3.0.html https://www.python.org/dev/peps/pep-3111/

raw_input() was renamed to input()

Fix tkinter crashing on MacOS by apply the following solution https://stackoverflow.com/questions/32019556/matplotlib-crashing-tkinter-application/34109240

Hi,

why in the mnist_4.2_batchnorm_convolutional.py file the biases are defined as constants? They are defined like this: B5 = tf.Variable(tf.constant(0.1, tf.float32, [10])) instead of the usual tf.Variable(tf.ones([10])/10).

And also, I refactored all your mnist training to tensorflow estimator format, you can find it here: https://github.com/ricoms/tensorflow-without-a-phd If you want I can create a pull request, although I deleted some of your files for visualization. For visualization, I am using a google Colaboratory ipynb that creates a tensorboard temporary url.

Another point is: my mnist_4.2_batchnorm_convolutional.py did not reach 99,5% accuracy. It's still below the mnist_3.2 results:

And I wonder if that is related to my first question, or my use of tf.layers.batch_normalization instead of your hardcoded function.

I intend to use it for a workshop at my college if it's ok with you. I will reference your work, of course. @martin-gorner and team, thanks for all the great material and tutorial videos! I'm a fan.

Thank you for your great video's, insights, codes, etc Much appreciated and inspiring. After Pong want to try other Atari games like Spaceinvaders, Pacman, etc and explore the boundaries of the policy gradient approach. Unfortunately got error when I simply replaced inner_env = gym.make('Pong-v0') to inner_env = gym.make('SpaceInvaders-v0'). Saw the code was quite resilient to capture any Atari game and the training also started but gave an error in "writing summary". Any pointers what might have caused it?

`args: {'n_epoch': 6000, 'batch_size': 10000, 'output_dir': '/tmp/pong_output', 'job_dir': '/tmp/pong_output', 'restore': False, 'render': False, 'save_checkpoint_steps': 1, 'learning_rate': 0.005, 'decay': 0.99, 'gamma': 0.99, 'laziness': 0.01, 'hidden_dim': 200, 'max_to_keep': 6000}

epoch 1 Rollout phase D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py:217: RuntimeWarning: invalid value encountered in true_divide prwd /= np.std(prwd) Train phase rollout reward: -4.323713773987632 Writing summary Traceback (most recent call last):

File "", line 1, in runfile('D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py', wdir='D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer')

File "D:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 668, in runfile execfile(filename, namespace)

File "D:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py", line 316, in main(args)

File "D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py", line 249, in main summary = sess.run(merged, feed_dict=feed_dict)

File "D:\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 877, in run run_metadata_ptr)

File "D:\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1100, in _run feed_dict_tensor, options, run_metadata)

File "D:\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1272, in _do_run run_metadata)

File "D:\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1291, in _do_call raise type(e)(node_def, op, message)

InvalidArgumentError: Nan in summary histogram for: summaries/train/model/dense_2/kernel [[Node: summaries/train/model/dense_2/kernel = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](summaries/train/model/dense_2/kernel/tag, summaries/train/model/dense_2/kernel/ReadVariableOp)]]

Caused by op 'summaries/train/model/dense_2/kernel', defined at: File "D:\anaconda\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "D:\anaconda\lib\runpy.py", line 85, in run_code exec(code, run_globals) File "D:\anaconda\lib\site-packages\spyder_kernels\console_main.py", line 11, in start.main() File "D:\anaconda\lib\site-packages\spyder_kernels\console\start.py", line 296, in main kernel.start() File "D:\anaconda\lib\site-packages\ipykernel\kernelapp.py", line 486, in start self.io_loop.start() File "D:\anaconda\lib\site-packages\tornado\platform\asyncio.py", line 127, in start self.asyncio_loop.run_forever() File "D:\anaconda\lib\asyncio\base_events.py", line 422, in run_forever self._run_once() File "D:\anaconda\lib\asyncio\base_events.py", line 1434, in _run_once handle._run() File "D:\anaconda\lib\asyncio\events.py", line 145, in _run self._callback(*self._args) File "D:\anaconda\lib\site-packages\tornado\platform\asyncio.py", line 117, in _handle_events handler_func(fileobj, events) File "D:\anaconda\lib\site-packages\tornado\stack_context.py", line 276, in null_wrapper return fn(*args, **kwargs) File "D:\anaconda\lib\site-packages\zmq\eventloop\zmqstream.py", line 450, in _handle_events self._handle_recv() File "D:\anaconda\lib\site-packages\zmq\eventloop\zmqstream.py", line 480, in _handle_recv self._run_callback(callback, msg) File "D:\anaconda\lib\site-packages\zmq\eventloop\zmqstream.py", line 432, in _run_callback callback(*args, **kwargs) File "D:\anaconda\lib\site-packages\tornado\stack_context.py", line 276, in null_wrapper return fn(*args, **kwargs) File "D:\anaconda\lib\site-packages\ipykernel\kernelbase.py", line 283, in dispatcher return self.dispatch_shell(stream, msg) File "D:\anaconda\lib\site-packages\ipykernel\kernelbase.py", line 233, in dispatch_shell handler(stream, idents, msg) File "D:\anaconda\lib\site-packages\ipykernel\kernelbase.py", line 399, in execute_request user_expressions, allow_stdin) File "D:\anaconda\lib\site-packages\ipykernel\ipkernel.py", line 208, in do_execute res = shell.run_cell(code, store_history=store_history, silent=silent) File "D:\anaconda\lib\site-packages\ipykernel\zmqshell.py", line 537, in run_cell return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs) File "D:\anaconda\lib\site-packages\IPython\core\interactiveshell.py", line 2662, in run_cell raw_cell, store_history, silent, shell_futures) File "D:\anaconda\lib\site-packages\IPython\core\interactiveshell.py", line 2785, in _run_cell interactivity=interactivity, compiler=compiler, result=result) File "D:\anaconda\lib\site-packages\IPython\core\interactiveshell.py", line 2907, in run_ast_nodes if self.run_code(code, result): File "D:\anaconda\lib\site-packages\IPython\core\interactiveshell.py", line 2961, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py', wdir='D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer') File "D:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 668, in runfile execfile(filename, namespace) File "D:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile exec(compile(f.read(), filename, 'exec'), namespace) File "D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py", line 316, in main(args) File "D:/OneDrive/Robotics/AINN/pong_google_cloud/tensorflow-without-a-phd/tensorflow-rl-spaceinvaders/trainer/task.py", line 148, in main tf.summary.histogram(var.op.name, var) File "D:\anaconda\lib\site-packages\tensorflow\python\summary\summary.py", line 187, in histogram tag=tag, values=values, name=scope) File "D:\anaconda\lib\site-packages\tensorflow\python\ops\gen_logging_ops.py", line 282, in histogram_summary "HistogramSummary", tag=tag, values=values, name=name) File "D:\anaconda\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "D:\anaconda\lib\site-packages\tensorflow\python\util\deprecation.py", line 454, in new_func return func(*args, **kwargs) File "D:\anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 3155, in create_op op_def=op_def) File "D:\anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 1717, in init self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Nan in summary histogram for: summaries/train/model/dense_2/kernel [[Node: summaries/train/model/dense_2/kernel = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](summaries/train/model/dense_2/kernel/tag, summaries/train/model/dense_2/kernel/ReadVariableOp)]]`

It is running on Anaconda on a Windows machine. Thank you.

Authorization Error Error 400: redirect_uri_mismatch

You can't sign in to this app because it doesn't comply with Google's OAuth 2.0 policy.

If you're the app developer, register the JavaScript origin in the Google Cloud Console.

Any guidance on how to do this?

Hi! I compared Keras fit time for dataset and experimental_distribute_dataset from your great notebook using latest TF version. It turned out that distributed dataset adds no speedup. Are you sure that your distributed input pipeline is well optimized for TPU? Why don't you use other optimizations like these:

def input_fn(batch_size):
    """> 2000 images/sec"""
    files = tf.data.Dataset.list_files(FLAGS.data_dir)

    def tftecord_dataset(filename):
        buffer_size = 8 * 1024 * 1024   # 8 MiB per file
        return tf.data.TFRecordDataset(filename, buffer_size=buffer_size)

    dataset = files.apply(tf.contrib.data.parallel_interleave(
        tftecord_dataset, cycle_length=32, sloppy=True))
    dataset = dataset.apply(tf.contrib.data.shuffle_and_repeat(10000, NUM_EPOCHS))
    dataset = dataset.apply(tf.contrib.data.map_and_batch(
        parser_fn, batch_size, num_parallel_calls=4))
    return dataset.prefetch(4)

if FLAGS.use_tpu:
    # When using TPU, wrap the optimizer with CrossShardOptimizer which
    # handles synchronizarion details between different TPU cores.
    optimizer = tpu_optimizer.CrossShardOptimizer(optimizer)

Hi When I tried training in cloud using this link https://github.com/GoogleCloudPlatform/tensorflow-without-a-phd/tree/master/tensorflow-mnist-tutorial/mlengine

I got error Error when submitting the job (gcloud.ml-engine.jobs.submit.training) argument --job-dir: Must be of form gs://bucket/object

I specified the bucket storage here but still got an error. gcloud ml-engine jobs submit training jobXXX --job-dir gs:///jobs/jobXXX --project --config config.yaml --module-name trainer.task --package-path trainer --runtime-version 1.4

Kindly help..thanks

Hi Martin,

I have the following issue while compiling the code. please let me know how to resolve this.

when I run the python command

python mnist_1.0_softmax.py I'm getting this error AttributeError: module 'tensorflow' has no attribute 'set_random_seed'

Method plot_learning_rate was not using the lr_func parameter and was using the global definition of lr_decay instead. This PR fixes it by using the correct method parameter.