discussion https://github.com/ibab/tensorflow-wavenet/issues/83 https://github.com/ibab/tensorflow-wavenet/issues/219

also we could then use gumpel-trick to reparametrize categorical dist and allow unrolling during training.

in issue 83 above, note that they speak about a larger kernel for the initial conv.

early support in image-support branch. v48 on gpus. seems to be a quite nice model so far. R=347, global condition=class,

top row: observed 392 px bottom row: generated 392px

top row: observed 200 px bottom row: generated 584 px

top row: observed 1 px bottom row: generated 783 px

top row: observed 200 px bottom row: generated 584 px

an attempt to tell the model where it is in the sequence. Good start would be transformers positional encoding procedure.

See e.g https://kazemnejad.com/blog/transformer_architecture_positional_encoding/

We could at the positional embedding as local conditioning, which would be required first.

consider DTW /FastDTW to account for phase-shifts

https://medium.com/walmartglobaltech/time-series-similarity-using-dynamic-time-warping-explained-9d09119e48ec https://github.com/slaypni/fastdtw/blob/master/fastdtw/fastdtw.py

allows us to employ reparametrization trick on categoricals and hence backprop through multiple steps that require sampling. see https://www.youtube.com/watch?v=JFgXEbgcT7g

Paper also describes local and global conditioning. In time series prediction we can treat these similar to positional encoding, i.e. season, time, etc. This would be similar to feeding in time as second channel input?

https://stats.stackexchange.com/questions/388130/wavenet-global-and-local-conditioning

if self.histograms: layer = 'layer{}'.format(layer_index) tf.histogram_summary(layer + '_filter', weights_filter) tf.histogram_summary(layer + '_gate', weights_gate) tf.histogram_summary(layer + '_dense', weights_dense) tf.histogram_summary(layer + '_skip', weights_skip)

this seems to cause problems when trying to actually perform a fit.

python -m autoregressive.scripts.train fit --config models\fseries_q127\config.yaml
usage: train.py [-h] [--config CONFIG] [--print_config [={comments,skip_null}+]] {fit,validate,test,predict,tune} ...
train.py: error: Configuration check failed :: Key "fit.data" is required but not included in config object or its value is None.```

However

python -m autoregressive.scripts.train --config models\fseries_q127\config.yaml fit

works

With the most current version 

python -m autoregressive.scripts.train fit --data autoregressive.datasets.MNISTDataModule --print_config > config.yaml

 does not generate a `fit:`.

https://colab.research.google.com/github/spatialaudio/digital-signal-processing-lecture/blob/master/quantization/linear_uniform_characteristic.ipynb#scrollTo=Oc-w6mUgXiiP

I am attempting to use your wavenet implementation to model some climate data, where my condition vector changes with time. The code mentions only global conditioning is currently supported. What exactly does this mean from an architecture perspective?

PixelCNN++ argues that neighboring intensity usually correlate is not captured by the softmax distribution. Instead, they propose a mixture model consisting of the logitic distribution (like normal but with heavier tails). https://arxiv.org/pdf/1701.05517.pdf

currently we assume that what we get as input is what we will predict as output (just shifted). However, thinking towards other research areas it might make sense that we rework that more generally:

model
  input: BxIxT
  output: BxQxT

where I might match Q but does not have to. In the training we would then have code like the following

def training_step(batch): 
  inputs = batch['x']
  if 't' in batch:
    targets = batch['t'] # allows us to provide alternative targets
  elif I == Q:
    targets = inputs[..., 1:]
    inputs = inputs[..., :-1]
  else:
    raise ValueError(...)

  logits = self.forward(inputs)
  loss = ce(logits, targets)

what's more is that we need to think about input transformers. Currently we use one-hot encoding hardwired into the model. We might instead consider a differentiable input_transform that is given to the model upon initialization. This would allow us to use differentiable embedding strategies.