Hi, thanks for you good work. I am new in TSC and try to follow your work. But I am confued about the validation dataset code in this repo. It seems that some classifiers lack validation datasets and just use the test dataset instead, e.g., encoder and inception. In this way, how do I determine the best model to get the results of test datasets in TSC problem?

Thank you.

Hi,

For some datasets (i.e. DodgerLoopDay), I got an error when running FCN classifier. Actually, I got the error "OSError: Unable to open file (unable to open file: name = 'xxx/results/fcn/UCRArchive_2018/DodgerLoopDay/best_model.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)" It seems that there is no best_model.hdf5 saved.

Do you know how to manage/fix this issue?

Thank you

Hi,

Thank you so much for sharing the code.

I'm having some issues with using class activation map on a dataset trained using FCN:

x_train.reshape(x_train.shape[0], x_train.shape[1], 1)

ValueError: cannot reshape array of size 1272000 into shape (424,750,1)

My x_train has the size (343, 750, 4) in which I have 343 samples with 750 points and 4 channels. What am I doing wrong?

Thank you in advance

First of all, thank you for your marvellous publication and also for open-sourcing your code.

I've been trying out your code with my own datasets and came across this error when using the MCNN model:

Method:  my_archive tmp_bs16_binarySecond_offset_kmeans mcnn
C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\sklearn\preproces
sing\_encoders.py:368: FutureWarning: The handling of integer data will change i
n version 0.22. Currently, the categories are determined based on the range [0,
max(values)], while in the future they will be determined based on the unique va
lues.
If you want the future behaviour and silence this warning, you can specify "cate
gories='auto'".
In case you used a LabelEncoder before this OneHotEncoder to convert the categor
ies to integers, then you can now use the OneHotEncoder directly.
  warnings.warn(msg, FutureWarning)
Using TensorFlow backend.
________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to

================================================================
input_1 (InputLayer)            (None, 14, 1)        0

________________________________________________________________________________
input_2 (InputLayer)            (None, 7, 1)         0

________________________________________________________________________________
input_3 (InputLayer)            (None, 5, 1)         0

________________________________________________________________________________
input_4 (InputLayer)            (None, 4, 1)         0

________________________________________________________________________________
input_5 (InputLayer)            (None, 10, 1)        0

________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 14, 256)      256         input_1[0][0]

________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 7, 256)       256         input_2[0][0]

________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 5, 256)       256         input_3[0][0]

________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 4, 256)       256         input_4[0][0]

________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 10, 256)      256         input_5[0][0]

________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D)  (None, 2, 256)       0           conv1d_1[0][0]

________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D)  (None, 2, 256)       0           conv1d_2[0][0]

________________________________________________________________________________
max_pooling1d_3 (MaxPooling1D)  (None, 2, 256)       0           conv1d_3[0][0]

________________________________________________________________________________
max_pooling1d_4 (MaxPooling1D)  (None, 2, 256)       0           conv1d_4[0][0]

________________________________________________________________________________
max_pooling1d_5 (MaxPooling1D)  (None, 2, 256)       0           conv1d_5[0][0]

________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 2, 1280)      0           max_pooling1d_1
[0][0]
                                                                 max_pooling1d_2
[0][0]
                                                                 max_pooling1d_3
[0][0]
                                                                 max_pooling1d_4
[0][0]
                                                                 max_pooling1d_5
[0][0]
________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 2, 256)       256         concatenate_1[0
][0]
________________________________________________________________________________
max_pooling1d_6 (MaxPooling1D)  (None, 2, 256)       0           conv1d_6[0][0]

________________________________________________________________________________
flatten_1 (Flatten)             (None, 512)          0           max_pooling1d_6
[0][0]
________________________________________________________________________________
dense_1 (Dense)                 (None, 256)          131328      flatten_1[0][0]

________________________________________________________________________________
dense_2 (Dense)                 (None, 2)            514         dense_1[0][0]

===============================================================
Total params: 133,378
Trainable params: 133,378
Non-trainable params: 0
________________________________________________________________________________
Traceback (most recent call last):
  File "main.py", line 112, in <module>
    fit_classifier()
  File "main.py", line 41, in fit_classifier
    classifier.fit(x_train,y_train,x_test,y_test, y_true)
  File "C:\Users\<username>\Desktop\Abschlussprojekt\Analysis\dl-4-tsc-master
\classifiers\mcnn.py", line 454, in fit
    y_test,y_true,pool_factor,filter_size)
  File "C:\Users\<username>\Desktop\Abschlussprojekt\Analysis\dl-4-tsc-master
\classifiers\mcnn.py", line 281, in train
    cost_ij, accuracy = model.train_on_batch(x,y)
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\keras\eng
ine\training.py", line 1216, in train_on_batch
    self._make_train_function()
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\keras\eng
ine\training.py", line 509, in _make_train_function
    loss=self.total_loss)
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\keras\leg
acy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\keras\opt
imizers.py", line 505, in get_updates
    self.updates.append(K.update(m, m_t))
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\keras\bac
kend\tensorflow_backend.py", line 973, in update
    return tf.assign(x, new_x)
  File "C:\Users\<username>\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflo
w\python\ops\state_ops.py", line 222, in assign
    return ref.assign(value, name=name)
AttributeError: 'Tensor' object has no attribute 'assign'

I am not sure what exactly is causing this error. I have tried the MCNN model with the UCR/Adiac dataset and it worked fine. I can only imagine it being due to the different input dataset. I have used the same input dataset for other models of your study and it worked fine on them as well.

It would be much appreciated if you could lead me in the right direction. If you need any more information, please let me know.

Best regards,

First of all: Thank you very much for providing this code. It really helps a lot not to reinvent the wheel every time.

Unfortunately, I'm greeted with the following error whenever I attempt to run the "Encoder" model:

ValueError: Unknown layer: InstanceNormalization

I did not change anything in the way the layers are imported. Could you please have a look into this issue?

Hello

Thanks again for sharing your code.

I successfully applied your code on my datasets and found, when inspecting the results in e.g. df_best_model.csv and df_metrics, that you deem the model as the best model, by monitoring the loss function of the training set. For example, in the FCN model you are setting a ModelCheckpoint by monitoring the training loss. But in the mentioned metrics and your paper you rate the model by its validation accuracy, is that correct?

file_path = self.output_directory+'best_model.hdf5'

model_checkpoint = keras.callbacks.ModelCheckpoint(filepath=file_path, 
    monitor='loss', save_best_only=True)

self.callbacks = [reduce_lr,model_checkpoint]

hist = self.model.fit(x_train, y_train, batch_size=mini_batch_size, 
    epochs=nb_epochs, verbose=self.verbose, validation_data=(x_val,y_val), 
    callbacks=self.callbacks)

I have run through multiple model / dataset variations and found that sometimes Model A with parameters A is better, measured in training loss, than Model A with parameters B. However, in these cases Model A with parameters A overfitted and its validation loss is worse than that of Model A with parameters B. When measuring the performance of a model by its validation accuracy, wouldn't it be more accurate to set the ModelCheckpoint to monitor val_loss or val_acc instead of train_loss?

Thanks

Thank you for your works!

It's a great contribution for the study of time series classification. I'm very interested on your works but I noticed that the link to Multivariate time series datasets is now missing.

Can you help me fix that or give me another link？ Thank you very much

Hi, I read your paper (Deep learning for time series classification: a review ) these days . I am always confused about how to analyze the CD diagram. For example, the FIg.7 in your paper. There are connections between MLP, Time-CNN, and TWIESN, as well as between TWIESN and MCDCNN. That means there's no statistical difference between MLP, Time-CNN and TWIESN, and there's no statistical difference between TWIESN and MCDCNN. So, there is no statistical difference between MLP and MCDCNN? I doubt the correctness of this conclusion. Because these four methods are not connected by a single connection. But I do not know what the right conclusion is, could you please give me some help? Thanks a lot.

Hi, Your work is great. I have a little question, the results in your table seem to be different from the 'Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline' and I read the issue 'can't reproduce the results with FordB dataset'. So I suspect that the version of UCR is responsible for the difference. Originally, I thought UCR2018 simply increased the number of datasets. Now it does not seem that way, I do not know if the data sets in UCR 2015 were all reprocessed in UCR 2018. This may lead to differences in the classification accuracy of the same data set under the same network model. Thanks for your help.

Hi there,

congratulations for this amazing repository!

I'm trying to reproduce some of the results, but I was going crazy with the FordB dataset because I was obtaining a much lower accuracy with every model. Then I checked the results in the file: "dl-4-tsc/results/results-ucr-128.csv" (below a sample of that file)

They don't match with the results table, where you get 87.8 accuracy with the FCN model as instance in stead of 77.x that is shown in the results file.

Is that all right? Am I missing something?

Thanks in advance, Jorge

Hi Dr Fawaz, Thanks for providing the deep learning tsc codes. I am about to begin my work which is also on the similar track. So, i downloaded the code from your page https://github.com/hfawaz/dl-4-tsc and cofee data (both in .zip) from UCR archive to my machine and ran the coffee data as mentioned in this page under Code section. Could your please tell me how long this one line (python3 main.py TSC Coffee fcn _itr_8) code should take to execute? if each prerequisite is completed already. Thank you.

Regards, Vibha

I tried with my data, looks poor, I guess it is overfitting, can you please discuss what is wrong happening and what can be changed ?

https://colab.research.google.com/drive/1dSvzTLVk65i0Q0yUL_4O3xOoJILCz8wa?usp=sharing

WARNING:tensorflow:From C:\Users\MOTNOG\anaconda3\lib\site-packages\tensorflow\python\ops\init_ops.py:97: calling GlorotUniform.init (from tensorflow.python.ops.i nit_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor WARNING:tensorflow:From C:\Users\MOTNOG\anaconda3\lib\site-packages\tensorflow\python\ops\init_ops.py:97: calling Zeros.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor C:\Users\MOTNOG\anaconda3\lib\site-packages\sklearn\metrics_classification.py:1272: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels w ith no predicted samples. Use zero_division parameter to control this behavior. _warn_prf(average, modifier, msg_start, len(result))

great code thanks may you clarify : will it work for multivariate time series prediction both regression and classification 1 where all values are continues values weight height age target 1 56 160 34 1.2 2 77 170 54 3.5 3 87 167 43 0.7 4 55 198 72 0.5 5 88 176 32 2.3

2 or even will it work for multivariate time series where values are mixture of continues and categorical values for example 2 dimensions have continues values and 3 dimensions are categorical values

color        weight     gender  height  age  target 

1 black 56 m 160 34 yes 2 white 77 f 170 54 no 3 yellow 87 m 167 43 yes 4 white 55 m 198 72 no 5 white 88 f 176 32 yes

Hi, just a quick question. In twiesn.py, the fit method of the classifier asks for y_true and y_test but doesn't seem to use the latter for anything (it's used to define self.y_test but nothing else). Is there a difference between the two? I haven't looked in detail but I think there was a similar point in inception.py? Cheers

Hello, great work!

I would like to know how the code handles non-stationary timeseries (based on p-value criteria / Dickey-Fuller-Test). Is there any stationization involved? / or do i need to stationize experiment data beforehand?

best regards