Hello,

I've been following the chapter on VAE and have copy pasted the code for the VAE faces training. I've came across a few issues with the code which I have corrected, but unfortunately I've been stumped by this last one.

TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='Placeholder:0', description="created by layer 'tf.cast_39'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as tf.cond, tf.function, gradient tapes, or tf.map_fn. Keras Functional model construction only supports TF API calls that do support dispatching, such as tf.math.add or tf.reshape. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer call and calling that layer on this symbolic input/output.

I've narrowed it down to the custom loss function as it will run when replaced with MSE:

`def compile(self, learning_rate, r_loss_factor): self.learning_rate = learning_rate

    ### COMPILATION
    def vae_r_loss(y_true, y_pred):
        r_loss = K.mean(K.square(y_true - y_pred), axis = [1,2,3])
        return r_loss_factor * r_loss

    def vae_kl_loss(y_true, y_pred):
        kl_loss =  -0.5 * K.sum(1 + self.log_var - K.square(self.mu) - K.exp(self.log_var), axis = 1)
        return kl_loss

    def vae_loss(y_true, y_pred):
        r_loss = vae_r_loss(y_true, y_pred)
        kl_loss = vae_kl_loss(y_true, y_pred)
        return  r_loss + kl_loss

    optimizer = tf.keras.optimizers.Adam(lr=learning_rate)
    self.model.compile(optimizer=optimizer, loss = vae_loss,  metrics = [vae_r_loss, vae_kl_loss])#'mse')#

`

I've tried replacing the K backend with tf.math functions but this hasn't worked. I'm not entirely sure what the error message is trying to communicate though. I'd really like to know how to get this to work as I've been quite interested in being able to implement custom loss functions.

Thanks a lot in advance for your support.

Paul

Error in 03_03_vae_digits_train Notebook (TensorFlow2 branch)

ValueError: The model cannot be compiled because it has no loss to optimize.

When running the cell below:

vae.train(     
    x_train
#     x_train[:1000]
    , batch_size = BATCH_SIZE
    , epochs = EPOCHS
    , run_folder = RUN_FOLDER
    , print_every_n_batches = PRINT_EVERY_N_BATCHES
    , initial_epoch = INITIAL_EPOCH
)

I get the following error:

WARNING:tensorflow:Output output_1 missing from loss dictionary. We assume this was done on purpose. The fit and evaluate APIs will not be expecting any data to be passed to output_1.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-ef78d2d1df6e> in <module>
      6     , run_folder = RUN_FOLDER
      7     , print_every_n_batches = PRINT_EVERY_N_BATCHES
----> 8     , initial_epoch = INITIAL_EPOCH
      9 )

~/SageMaker/generative-deep-l/TF2/GDL_code/models/VAE.py in train(self, x_train, batch_size, epochs, run_folder, print_every_n_batches, initial_epoch, lr_decay)
    224             , epochs = epochs
    225             , initial_epoch = initial_epoch
--> 226             , callbacks = callbacks_list
    227         )
    228 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    823         max_queue_size=max_queue_size,
    824         workers=workers,
--> 825         use_multiprocessing=use_multiprocessing)
    826 
    827   def evaluate(self,

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    233           max_queue_size=max_queue_size,
    234           workers=workers,
--> 235           use_multiprocessing=use_multiprocessing)
    236 
    237       total_samples = _get_total_number_of_samples(training_data_adapter)

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_training_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, steps_per_epoch, validation_split, validation_data, validation_steps, shuffle, distribution_strategy, max_queue_size, workers, use_multiprocessing)
    591         max_queue_size=max_queue_size,
    592         workers=workers,
--> 593         use_multiprocessing=use_multiprocessing)
    594     val_adapter = None
    595     if validation_data:

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_inputs(model, mode, x, y, batch_size, epochs, sample_weights, class_weights, shuffle, steps, distribution_strategy, max_queue_size, workers, use_multiprocessing)
    644     standardize_function = None
    645     x, y, sample_weights = standardize(
--> 646         x, y, sample_weight=sample_weights)
    647   elif adapter_cls is data_adapter.ListsOfScalarsDataAdapter:
    648     standardize_function = standardize

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split, shuffle, extract_tensors_from_dataset)
   2376     is_compile_called = False
   2377     if not self._is_compiled and self.optimizer:
-> 2378       self._compile_from_inputs(all_inputs, y_input, x, y)
   2379       is_compile_called = True
   2380 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _compile_from_inputs(self, all_inputs, target, orig_inputs, orig_target)
   2634         sample_weight_mode=self.sample_weight_mode,
   2635         run_eagerly=self.run_eagerly,
-> 2636         experimental_run_tf_function=self._experimental_run_tf_function)
   2637 
   2638   # TODO(omalleyt): Consider changing to a more descriptive function name.

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, distribute, **kwargs)
    444 
    445       # Creates the model loss and weighted metrics sub-graphs.
--> 446       self._compile_weights_loss_and_weighted_metrics()
    447 
    448       # Functions for train, test and predict will

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _compile_weights_loss_and_weighted_metrics(self, sample_weights)
   1608       #                   loss_weight_2 * output_2_loss_fn(...) +
   1609       #                   layer losses.
-> 1610       self.total_loss = self._prepare_total_loss(masks)
   1611 
   1612   def _prepare_skip_target_masks(self):

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in _prepare_total_loss(self, masks)
   1707       if total_loss is None:
   1708         if not self.losses:
-> 1709           raise ValueError('The model cannot be compiled '
   1710                            'because it has no loss to optimize.')
   1711         else:

ValueError: The model cannot be compiled because it has no loss to optimize.

Hi, I'm trying to execute the notebooks as I am reading the book on O'Reilly platform but the utils folder (which correspond to a module used in chapter 3) is empty. Is it too early to try execute the notebooks? Thanks.

The Large-scale CelebFaces Attributes (CelebA) Dataset is available as a Google Drive folder link. It is not possible to downlaod it via standard tools such as wget.

Downloading each file manually is time consuming. Could you pls. help how we could download the dataset, preferrably via Command Line, and with automation? Also, it would be great to explicitly state which files to download (it seems aligned&cropped_images dataset shall be downloaded, do you agree? If yes, could we pls. specify in the notebook code to avoid ambiguity?)

Thank you.

Please note that I am not a very good programmer. I understand that my doubt might be silly to a few people.

Why has the author made a separate file for models? [If I am not wrong, 'models' is not actually a well-known package but a file that contains a lot of other python files that contain classes we need to perform the task. Considering, I am still on Chapter 3, models' folder's AE.py file has the class Autoencoder, which has all the custom functions required to build the model].

Hi David, I am enjoying reading your book, I am around page 115.

But something that is annoying me a lot is that not all the code from the book is in the repo. For example now I wanted to copy and rewrite the GAN code, but it's nowhere to be seen, and you only show how to use the GAN class, while all the other code you showed about how to build a GAN from scratch is not part of the notebook.

It might be only my opinion, but when I am reading a programming book, I expect every bit of code to be available and tested, because that's how people learns. "Hands on machine learning" is an example of this.

Will all the code from the book be part of the notebooks?

I set up the conda environment according to instructions; all notebooks in chapter 2 work, but as soon as I get to chapter 3 I'm stopped by this error in 03_01_autoencoder_train.ipynb and 03_03_vae_digits_train.ipynb:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
~/anaconda3/envs/generative/lib/python3.6/site-packages/pydot.py in create(self, prog, format, encoding)
   1914                 arguments=arguments,
-> 1915                 working_dir=tmp_dir,
   1916             )

~/anaconda3/envs/generative/lib/python3.6/site-packages/pydot.py in call_graphviz(program, arguments, working_dir, **kwargs)
    135         stdout=subprocess.PIPE,
--> 136         **kwargs
    137     )

~/anaconda3/envs/generative/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
    728                                 errread, errwrite,
--> 729                                 restore_signals, start_new_session)
    730         except:

~/anaconda3/envs/generative/lib/python3.6/subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session)
   1363                             err_msg += ': ' + repr(err_filename)
-> 1364                     raise child_exception_type(errno_num, err_msg, err_filename)
   1365                 raise child_exception_type(err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'dot': 'dot'

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
~/anaconda3/envs/generative/lib/python3.6/site-packages/keras/utils/vis_utils.py in _check_pydot()
     25         # to check the pydot/graphviz installation.
---> 26         pydot.Dot.create(pydot.Dot())
     27     except OSError:

~/anaconda3/envs/generative/lib/python3.6/site-packages/pydot.py in create(self, prog, format, encoding)
   1921                     prog=prog)
-> 1922                 raise OSError(*args)
   1923             else:

FileNotFoundError: [Errno 2] "dot" not found in path.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-4-2cbc3320019b> in <module>
     11 
     12 if MODE == 'build':
---> 13     AE.save(RUN_FOLDER)
     14 else:
     15     AE.load_weights(os.path.join(RUN_FOLDER, 'weights/weights.h5'))

~/repo/GDL_code/models/AE.py in save(self, folder)
    153                 ], f)
    154 
--> 155         self.plot_model(folder)
    156 
    157 

~/repo/GDL_code/models/AE.py in plot_model(self, run_folder)
    182 
    183     def plot_model(self, run_folder):
--> 184         plot_model(self.model, to_file=os.path.join(run_folder ,'viz/model.png'), show_shapes = True, show_layer_names = True)
    185         plot_model(self.encoder, to_file=os.path.join(run_folder ,'viz/encoder.png'), show_shapes = True, show_layer_names = True)
    186         plot_model(self.decoder, to_file=os.path.join(run_folder ,'viz/decoder.png'), show_shapes = True, show_layer_names = True)

~/anaconda3/envs/generative/lib/python3.6/site-packages/keras/utils/vis_utils.py in plot_model(model, to_file, show_shapes, show_layer_names, rankdir)
    130             'LR' creates a horizontal plot.
    131     """
--> 132     dot = model_to_dot(model, show_shapes, show_layer_names, rankdir)
    133     _, extension = os.path.splitext(to_file)
    134     if not extension:

~/anaconda3/envs/generative/lib/python3.6/site-packages/keras/utils/vis_utils.py in model_to_dot(model, show_shapes, show_layer_names, rankdir)
     53     from ..models import Sequential
     54 
---> 55     _check_pydot()
     56     dot = pydot.Dot()
     57     dot.set('rankdir', rankdir)

~/anaconda3/envs/generative/lib/python3.6/site-packages/keras/utils/vis_utils.py in _check_pydot()
     27     except OSError:
     28         raise OSError(
---> 29             '`pydot` failed to call GraphViz.'
     30             'Please install GraphViz (https://www.graphviz.org/) '
     31             'and ensure that its executables are in the $PATH.')

OSError: `pydot` failed to call GraphViz.Please install GraphViz (https://www.graphviz.org/) and ensure that its executables are in the $PATH.

I did some searching and people are saying to fix it by running conda install graphviz followed by conda install python-graphviz, but I tried that and it doesn't change anything. The book doesn't mention GraphViz anywhere.

My pip (1.19.2) could not find Tensorflow 1.12.0 as specified in requirements.txt. After installing lowest version I could find (1.13.1) Tensorboard was incompatible - after several tries I was obliged to use 1.14.0 for both - the further checks in chapter1 worked fine. However will I perhaps have trouble running some of the book examples? I don't mind the version discrepancy otherwise - just wanted to report the issue in case someone else may also run into the problem.

I am using Python 3.7.2 on Windows 10.

Heelo,

I was trying to run 03_01 code. However, I got the following error from the second cell.

run params

SECTION = 'vae' RUN_ID = '0001' DATA_NAME = 'digits' RUN_FOLDER = '/run/{}/'.format(SECTION) RUN_FOLDER += '_'.join([RUN_ID, DATA_NAME])

if not os.path.exists(RUN_FOLDER): os.mkdir(RUN_FOLDER) os.mkdir(os.path.join(RUN_FOLDER, 'viz')) os.mkdir(os.path.join(RUN_FOLDER, 'images')) os.mkdir(os.path.join(RUN_FOLDER, 'weights'))

MODE = 'build' #'load' #

FileNotFoundError Traceback (most recent call last) in 7 8 if not os.path.exists(RUN_FOLDER): ----> 9 os.mkdir(RUN_FOLDER) 10 os.mkdir(os.path.join(RUN_FOLDER, 'viz')) 11 os.mkdir(os.path.join(RUN_FOLDER, 'images'))

FileNotFoundError: [Errno 2] No such file or directory: './run/vae/0001_digits'

Hey great read and great code. I stumbled on some tiny issues when running the GAN: in def train_discriminator the:

else:
            idx = np.random.randint(0, x_train.shape[0], batch_size)
            true_imgs = x_train[idx]

should be

else:
           idx = np.random.randint(0, x_train[0].shape[0], batch_size)
            true_imgs = x_train[0][idx]

as it comes back with x at [0] and y at [1] when running the train I had to use:

graph = tf.get_default_graph()
with graph.as_default():
     gan.train(........

as it was complaining about tensors not being part of the graph. that seems to be a Keras issue.

Running the vanilla notebook, I get the following at the train.fit step:

TypeError Traceback (most recent call last) Cell In[10], line 1 ----> 1 vae.train(
2 x_train 3 , batch_size = BATCH_SIZE 4 , epochs = EPOCHS 5 , run_folder = RUN_FOLDER 6 , print_every_n_batches = PRINT_EVERY_N_BATCHES 7 , initial_epoch = INITIAL_EPOCH 8 )

File ~/projects/generative_book/GDL_code/models/VAE.py:195, in VariationalAutoencoder.train(self, x_train, batch_size, epochs, run_folder, print_every_n_batches, initial_epoch, lr_decay) 191 checkpoint2 = ModelCheckpoint(os.path.join(run_folder, 'weights/weights.h5'), save_weights_only = True, verbose=1) 193 callbacks_list = [checkpoint1, checkpoint2, custom_callback, lr_sched] --> 195 self.model.fit(
196 x_train 197 , x_train 198 , batch_size = batch_size 199 , shuffle = True 200 , epochs = epochs 201 , initial_epoch = initial_epoch 202 , callbacks = callbacks_list 203 )

File ~/miniforge3/envs/generative/lib/python3.10/site-packages/keras/utils/traceback_utils.py:70, in filter_traceback..error_handler(*args, **kwargs) 67 filtered_tb = _process_traceback_frames(e.traceback) 68 # To get the full stack trace, call: 69 # tf.debugging.disable_traceback_filtering() ---> 70 raise e.with_traceback(filtered_tb) from None 71 finally: 72 del filtered_tb

File /var/folders/x0/8d985bj50sgd0mgq51l3m0d40000gp/T/autograph_generated_filev67pwrag.py:15, in outer_factory..inner_factory..tf__train_function(iterator) 13 try: 14 do_return = True ---> 15 retval = ag_.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope) 16 except: 17 do_return = False

TypeError: in user code:

File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/training.py", line 1160, in train_function  *
    return step_function(self, iterator)
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/training.py", line 1146, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/training.py", line 1135, in run_step  **
    outputs = model.train_step(data)
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/training.py", line 994, in train_step
    loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/training.py", line 1052, in compute_loss
    return self.compiled_loss(
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 317, in __call__
    self._total_loss_mean.update_state(
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/utils/metrics_utils.py", line 77, in decorated
    update_op = update_state_fn(*args, **kwargs)
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/metrics/base_metric.py", line 143, in update_state_fn
    return ag_update_state(*args, **kwargs)
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/metrics/base_metric.py", line 486, in update_state  **
    sample_weight = tf.__internal__.ops.broadcast_weights(
File "/Users/jadolfbr/miniforge3/envs/generative/lib/python3.10/site-packages/keras/engine/keras_tensor.py", line 283, in __array__
    raise TypeError(

TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='Placeholder:0', description="created by layer 'tf.cast_2'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or `tf.map_fn`. Keras Functional model construction only supports TF API calls that *do* support dispatching, such as `tf.math.add` or `tf.reshape`. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer `call` and calling that layer on this symbolic input/output.

I am unable to find the yaml file to follow along with the book. Kindly let us know if it is intended and will be added at a later date or if I am missing something.

When training the batch normalized model, it throws this error:

InvalidArgumentError: Graph execution error:
...
    File "/home/name/.local/lib/python3.8/site-packages/keras/layers/normalization/batch_normalization.py", line 571, in _fused_batch_norm_training
      return tf.compat.v1.nn.fused_batch_norm(
Node: 'model_4/batch_normalization_14/FusedBatchNormV3'
DML doesn't support exponential_avg_factor != 1 at the moment
	 [[{{node model_4/batch_normalization_14/FusedBatchNormV3}}]] [Op:__inference_train_function_85820]

I'm using Tensorflow 2.9.1 and I'm on the tensorflow_2 branch.

Getting the following error , when trying to run vae.train

` TypeError: in user code:

File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1051, in train_function  *
    return step_function(self, iterator)
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1040, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1030, in run_step  **
    outputs = model.train_step(data)
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 890, in train_step
    loss = self.compute_loss(x, y, y_pred, sample_weight)
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 948, in compute_loss
    return self.compiled_loss(
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\compile_utils.py", line 239, in __call__
    self._loss_metric.update_state(
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\utils\metrics_utils.py", line 70, in decorated
    update_op = update_state_fn(*args, **kwargs)
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\metrics\base_metric.py", line 140, in update_state_fn
    return ag_update_state(*args, **kwargs)
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\metrics\base_metric.py", line 449, in update_state  **
    sample_weight = tf.__internal__.ops.broadcast_weights(
File "C:\Users\arind\anaconda3\envs\tf\lib\site-packages\keras\engine\keras_tensor.py", line 254, in __array__
    raise TypeError(

TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='Placeholder:0', description="created by layer 'tf.cast_2'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or `tf.map_fn`. Keras Functional model construction only supports TF API calls that *do* support dispatching, such as `tf.math.add` or `tf.reshape`. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer `call` and calling that layer on this symbolic input/output.

` Edit-1 25-Aug-22

I am certain that the error is coming from the custom KL Loss function. In the compile method , if I use only the vae_r_loss, the code works

` def vae_r_loss(y_true, y_pred): r_loss = K.mean(K.square(y_true - y_pred), axis = [1,2,3]) return r_loss_factor * r_loss

def vae_kl_loss(y_true, y_pred):
    kl_loss =  -0.5 * K.sum(1 + self.log_var - K.square(self.mu) - K.exp(self.log_var), axis = 1)
    return kl_loss

def vae_loss(y_true, y_pred):
    r_loss = vae_r_loss(y_true, y_pred)
    kl_loss = vae_kl_loss(y_true, y_pred)
    return  r_loss + kl_loss

optimizer = Adam(lr=learning_rate)
self.model.compile(optimizer=optimizer, loss = vae_r_loss,  metrics = [vae_r_loss])

`

In translation of book about deep learning and generative modelling it is said that in WGAN-GP models we should cut normalisation layers. This is strange because i can still see them on github page is it a mistake or not?

I am wondering how _build_adversarial method works. I see that This method is called only once during WGANGP init. I keep asking myself how line : self.set_trainable(self.generator, False) can work every iteration if this method is called only once And also how RandomWeightedAverage images can be counted every iteration?

Can somebody explain this to me? This is probably some functionality of Keras but I cant see this