An LSTM for time-series classification

Related tags

Deep Learning LSTM_tsc
Overview

Update 10-April-2017

And now it works with Python3 and Tensorflow 1.1.0

Update 02-Jan-2017

I updated this repo. Now it works with Tensorflow 0.12. In this readme I comment on some new benchmarks

LSTM for time-series classification

This post implements a Long Short-term memory for time series classification(LSTM). An LSTM is the extension of the classical Recurrent Neural Network. It has more flexibility and interpretable features such as a memory it can read, write and forget.

Aim

This repo aims to show the minimal Tensorflow code for proper time series classification. The main function loads the data and iterates over training steps. The tsc_model.py scripts contains the actual model. This repo contrasts with another project where I implement a similar script using convolutional neural networks as the model

Data and results

The code generalizes for any of the UCR time series. With the parameter dataset you can run the code on any of their datasets. For your interests, you may compare performances with the nice overview in this paper. They benchmark their CNN and other models on many of the UCR time series datasets This code works amongst others for

  • Two_Patterns where it achieves state-of-the-art, bein 100% test accuracy
  • ChlorineConcentration where it achieves state-of-the-art, being 80% test accuracy

Credits

Credits for this project go to Tensorflow for providing a strong example, the UCR archive for the dataset and my friend Ryan for strong feedback.

License is in LICENSE.txt

As always, I am curious to any comments and questions. Reach me at [email protected]

Comments
  • TypeError 'The value of a feed cannot be a tf.Tensor object. '

    TypeError 'The value of a feed cannot be a tf.Tensor object. '

    raise type error with tensorflow 0.11rc0; numpy 1.11.1; python 2.7.12; macOS 10.12;

    Train with approximately 60 epochs tsc_main.py:23: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future X_batch = X_train[ind_N,ind_start:ind_start+num_steps] At 0 out of 2000: Cost is TRAIN1.734(1.390) VAL1.599 and val acc is 0.400 At 100 out of 2000: Cost is TRAIN1.380(1.387) VAL1.376 and val acc is 0.300 At 200 out of 2000: Cost is TRAIN1.288(1.339) VAL1.159 and val acc is 0.733 At 300 out of 2000: Cost is TRAIN1.010(1.171) VAL1.106 and val acc is 0.567 At 400 out of 2000: Cost is TRAIN0.935(1.044) VAL1.041 and val acc is 0.533 At 500 out of 2000: Cost is TRAIN1.058(0.982) VAL0.776 and val acc is 0.633 At 600 out of 2000: Cost is TRAIN0.774(0.908) VAL1.153 and val acc is 0.633 At 700 out of 2000: Cost is TRAIN0.823(0.866) VAL0.643 and val acc is 0.833 At 800 out of 2000: Cost is TRAIN0.626(0.814) VAL0.876 and val acc is 0.633 At 900 out of 2000: Cost is TRAIN1.140(0.793) VAL0.792 and val acc is 0.600 At 1000 out of 2000: Cost is TRAIN0.800(0.753) VAL0.835 and val acc is 0.733 At 1100 out of 2000: Cost is TRAIN0.890(0.746) VAL0.571 and val acc is 0.767 At 1200 out of 2000: Cost is TRAIN0.634(0.725) VAL1.061 and val acc is 0.400 At 1300 out of 2000: Cost is TRAIN0.851(0.715) VAL0.550 and val acc is 0.767 At 1400 out of 2000: Cost is TRAIN0.504(0.694) VAL0.682 and val acc is 0.600 At 1500 out of 2000: Cost is TRAIN0.647(0.706) VAL0.870 and val acc is 0.567 At 1600 out of 2000: Cost is TRAIN0.490(0.702) VAL0.430 and val acc is 0.800 At 1700 out of 2000: Cost is TRAIN0.585(0.695) VAL0.548 and val acc is 0.767 At 1800 out of 2000: Cost is TRAIN1.013(0.676) VAL0.638 and val acc is 0.800 At 1900 out of 2000: Cost is TRAIN0.937(0.649) VAL0.817 and val acc is 0.600 tsc_main.py:31: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future test_acc = np.zeros(num_batch) Traceback (most recent call last): File "tsc_main.py", line 178, in acc_test = check_test(X_test,y_test,batch_size,num_steps) File "tsc_main.py", line 34, in check_test test_acc[i] = sess.run(accuracy,feed_dict = {input_data: X_batch, targets: y_batch, initial_state:state,keep_prob:1}) File "/usr/local/var/pyenv/versions/2.7.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run run_metadata_ptr) File "/usr/local/var/pyenv/versions/2.7.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 875, in _run raise TypeError('The value of a feed cannot be a tf.Tensor object. ' TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, or numpy ndarrays.

    opened by ghost 4
  • TypeError: Fetch argument None has invalid type <class 'NoneType'>

    TypeError: Fetch argument None has invalid type

    Hi, first thing: your project is very interesting. Second: I got an error at the execution at

    summary = sess.run( merged , {input_data: nextBatch, labels: nextBatchLabels})

    Do you have any ideas how to fix it ?

    Thank you in advance

    Traceback (most recent call last): File "Oriole+LSTM.py", line 220, in summary = sess.run( merged , {input_data: nextBatch, labels: nextBatchLabels}) File "C:\Anaconda\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 789, in run run_metadata_ptr) File "C:\Anaconda\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 984, in _run self._graph, fetches, feed_dict_string, feed_handles=feed_handles) File "C:\Anaconda\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 410, in init self._fetch_mapper = _FetchMapper.for_fetch(fetches) File "C:\Anaconda\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 227, in for_fetch (fetch, type(fetch))) TypeError: Fetch argument None has invalid type <class 'NoneType'>

    opened by marleymwangi 1
  • will it work for multivariate time series prediction both regression and classification

    will it work for multivariate time series prediction both regression and classification

    great code thanks may you clarify : will it work for multivariate time series prediction both regression and classification 1 where all values are continues values weight height age target 1 56 160 34 1.2 2 77 170 54 3.5 3 87 167 43 0.7 4 55 198 72 0.5 5 88 176 32 2.3

    2 or even will it work for multivariate time series where values are mixture of continues and categorical values for example 2 dimensions have continues values and 3 dimensions are categorical values

    color        weight     gender  height  age  target

    1 black 56 m 160 34 yes 2 white 77 f 170 54 no 3 yellow 87 m 167 43 yes 4 white 55 m 198 72 no 5 white 88 f 176 32 yes

    opened by Sandy4321 2
  • loadtxt “could not convert string to float”

    loadtxt “could not convert string to float”

    in the 'tsc_model', data_train = np.loadtxt(datadir+'_TRAIN.txt',delimiter=',')

    The result is 'could not convert string to float:' I don't know what is the reason. Thank you

    opened by stef729 0
  • Tensor Flow 1.5 needs the following code modifications to work.

    Tensor Flow 1.5 needs the following code modifications to work.

    in tsc_model.py

    from::

    from tensorflow.contrib.rnn.python.ops import core_rnn

    to::

    import tensorflow.contrib.rnn as core_rnn

    No check_test in model.py

    In tsc_main .py , from::

    from tsc_model import Model,sample_batch,load_data,check_test

    to::

    from tsc_model import Model,sample_batch,load_data

    Some Datasets like ECG200 have Negative class labels which makes the softmax classifier fail.

    opened by command-paul 0
  • AUC calculation

    AUC calculation

    Hi! Thanks for your code. In addition to accuracy, I am trying to obtain AUC values. I am suspecting that I am doing something wrong in that accuracy results are good but AUC are below 0.5.

    This is what I have in the tsc_model.py file: a = tf.cast(tf.argmax(logits, 1),tf.float32) #AUC b = tf.cast(self.labels,tf.float32) #AUC auc = tf.contrib.metrics.streaming_auc(a,b) #AUC self.auc_value = tf.reduce_mean(tf.cast(auc, "float")) #AUC**

    Then, in the main program, I have: sess.run(tf.initialize_local_variables()) #for AUC auc_test = sess.run([model.auc_value],feed_dict = {model.input: Xtest_batch, model.labels: ytest_batch, model.keep_prob:1.0}) print('Result of testing: AUC is %5.3f'%(auc_test))

    ***Please let me know if you spot an incorrect approach. Thanks!

    opened by nmsommer 0
  • cannot import name 'check_test'

    cannot import name 'check_test'

    In the commit Make compatible with python3 and TF1.1.0 you deleted the function check_test in the tsc_model.py, but forgot to delete in the code from tsc_model import Model,sample_batch,load_data,check_test in the tsc_main.py

    opened by ustcjerry 1
  • download UCR

    download UCR

    thank you for this awesome tutorial,now,I try to run it,however,when I read the full pdf provided by UCRArchive ,still can't find the password to unzip data.would you give me an obvious hint?

    opened by Twinkle123321 2
Owner
Rob Romijnders
AI resident @ Google Brain
Rob Romijnders
GitHub
MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification

MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification

null 187 Dec 26, 2022
tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai

timeseriesAI 2.8k Jan 8, 2023
ICML 21 - Voice2Series: Reprogramming Acoustic Models for Time Series Classification

Voice2Series-Reprogramming Voice2Series: Reprogramming Acoustic Models for Time Series Classification International Conference on Machine Learning (IC

null 49 Jan 3, 2023
Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.

Framework overview This library allows to quickly implement different architectures based on Reservoir Computing (the family of approaches popularized

Filippo Bianchi 249 Dec 21, 2022
A real world application of a Recurrent Neural Network on a binary classification of time series data

What is this This is a real world application of a Recurrent Neural Network on a binary classification of time series data. This project includes data

Josep Maria Salvia Hornos 2 Jan 30, 2022
Using LSTM write Tang poetry

本教程将通过一个示例对LSTM进行介绍。通过搭建训练LSTM网络，我们将训练一个模型来生成唐诗。本文将对该实现进行详尽的解释，并阐明此模型的工作方式和原因。并不需要过多专业知识，但是可能需要新手花一些时间来理解的模型训练的实际情况。为了节省时间，请尽量选择GPU进行训练。

null 56 Dec 15, 2022
OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network

Stock Price Prediction of Apple Inc. Using Recurrent Neural Network OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network Dataset:

Nouroz Rahman 410 Jan 5, 2023
A resource for learning about deep learning techniques from regression to LSTM and Reinforcement Learning using financial data and the fitness functions of algorithmic trading

A tour through tensorflow with financial data I present several models ranging in complexity from simple regression to LSTM and policy networks. The s

null 195 Dec 7, 2022
Using multidimensional LSTM neural networks to create a forecast for Bitcoin price

Multidimensional LSTM BitCoin Time Series Using multidimensional LSTM neural networks to create a forecast for Bitcoin price. For notes around this co

Jakob Aungiers 318 Dec 14, 2022
Multi-layer convolutional LSTM with Pytorch

Convolution_LSTM_pytorch Thanks for your attention. I haven't got time to maintain this repo for a long time. I recommend this repo which provides an

Zijie Zhuang 734 Jan 3, 2023
Incorporating Transformer and LSTM to Kalman Filter with EM algorithm

Deep learning based state estimation: incorporating Transformer and LSTM to Kalman Filter with EM algorithm Overview Kalman Filter requires the true p

zshicode 57 Dec 27, 2022
Forecasting directional movements of stock prices for intraday trading using LSTM and random forest

Forecasting directional movements of stock-prices for intraday trading using LSTM and random-forest https://arxiv.org/abs/2004.10178 Pushpendu Ghosh,

Pushpendu Ghosh 270 Dec 24, 2022
Deep learning based hand gesture recognition using LSTM and MediaPipie.

Hand Gesture Recognition Deep learning based hand gesture recognition using LSTM and MediaPipie. Demo video using PingPong Robot Files Pretrained mode

Brad 24 Nov 11, 2022
PyTorch implementation of the Quasi-Recurrent Neural Network - up to 16 times faster than NVIDIA's cuDNN LSTM

Quasi-Recurrent Neural Network (QRNN) for PyTorch Updated to support multi-GPU environments via DataParallel - see the the multigpu_dataparallel.py ex

Salesforce 1.3k Dec 28, 2022
A3C LSTM Atari with Pytorch plus A3G design

NEWLY ADDED A3G A NEW GPU/CPU ARCHITECTURE OF A3C FOR SUBSTANTIALLY ACCELERATED TRAINING!! RL A3C Pytorch NEWLY ADDED A3G!! New implementation of A3C

David Griffis 532 Jan 2, 2023
Tree LSTM implementation in PyTorch

Tree-Structured Long Short-Term Memory Networks This is a PyTorch implementation of Tree-LSTM as described in the paper Improved Semantic Representati

Riddhiman Dasgupta 529 Dec 10, 2022
Sign Language is detected in realtime using video sequences. Our approach involves MediaPipe Holistic for keypoints extraction and LSTM Model for prediction.

RealTime Sign Language Detection using Action Recognition Approach Real-Time Sign Language is commonly predicted using models whose architecture consi

Rishikesh S 15 Aug 20, 2022
LSTM and QRNN Language Model Toolkit for PyTorch

LSTM and QRNN Language Model Toolkit This repository contains the code used for two Salesforce Research papers: Regularizing and Optimizing LSTM Langu

Salesforce 1.9k Jan 8, 2023
Multi-layer convolutional LSTM with Pytorch

Convolution_LSTM_pytorch Thanks for your attention. I haven't got time to maintain this repo for a long time. I recommend this repo which provides an

Zijie Zhuang 733 Dec 30, 2022