PyTorch GPU implementation of the ES-RNN model for time series forecasting

Overview

Fast ES-RNN: A GPU Implementation of the ES-RNN Algorithm

A GPU-enabled version of the hybrid ES-RNN model by Slawek et al that won the M4 time-series forecasting competition by a large margin. The details of our implementation and the results are discussed in detail on this paper

Getting Started

Prerequisites

Python (3.5+)
Tensorflow (1.12+ to 1.14)
PyTorch (0.4.1)
Zalando Research's Dilated RNN

Dataset

Please download the M4 competition dataset directly from here and put the files in the data directory.

Running the algorithm

Either use an IDE such as PyCharm or make sure to add the es_rnn folder to your PYTHON PATH before running the main.py in the es_rnn folder. You can change the configurations of the algorithm in the config.py file.

Built With

  • Python - The data science language ;)
  • PyTorch - The dynamic framework for computation

Authors

License

This project is licensed under the MIT License - see the LICENSE file for details

Acknowledgments

  • Thank you to the original author of the algorithm Smyl Slawek slaweks17 for advice and for creating this amazing algorithm
  • Zalando Research zalandoresearch for their implementation of Dilated RNN

Citation

If you choose to use our implementation in your work please cite us as:

@article{ReddKhinMarini,
       author = {{Redd}, Andrew and {Khin}, Kaung and {Marini}, Aldo},
        title = "{Fast ES-RNN: A GPU Implementation of the ES-RNN Algorithm}",
      journal = {arXiv e-prints},
         year = "2019",
        month = "Jul",
          eid = {arXiv:1907.03329},
        pages = {arXiv:1907.03329},
archivePrefix = {arXiv},
       eprint = {1907.03329},
 primaryClass = {cs.LG}
}

Comments
  • Poor Performance on GCP V100s

    Poor Performance on GCP V100s

    I'm running the code nearly unchanged on a Google Cloud Compute instance with 2x Nvidia V100, 60GB RAM, 16CPUs. config.py is unchanged.

    With 15 epochs on the Quarterly data, total training time is 16.01 minutes, almost double the 8.94minutes shown in the paper. However, the validation results at the end of epoch 15 are nearly identical to the paper's reported results:

    {'Demographic': 10.814908027648926, 'Finance': 10.71678638458252, 'Industry': 7.436440944671631, 'Macro': 9.547700881958008, 'Micro': 11.63847827911377, 'Other': 7.911505699157715, 'Overall': 10.091866493225098, 'loss': 7.8162946701049805}
    

    When I remove the model saving step, training time decreased to 15.76 minutes.

    I downloaded the dataset from the provided link, and made no changes.

    I'm using updated package versions, although I wouldn't expect this to halve performance:

    • pytorch 1.2
    • tensorflow 1.14.0

    What hardware configuration was the authors' testing done on? I'm using dual V100s, the highest-end GPUs available on GCP. I'd expect to match or outperform the reported benchmarks. Do you have any thoughts on why my performance is considerably worse in my situation?

    opened by xanderdunn 3
  • Questions/Clarifications on ESRNN-GPU vs original ES-RNN

    Questions/Clarifications on ESRNN-GPU vs original ES-RNN

    Great work on this project! Having used a version of the original ES-RNN code, I have a few questions about the differences between the implementations and the results presented in the paper.

    1. The paper mentions "Note that for monthly data, Smyl et al. (2018) were running the algorithm of 6 pairs of 2 workers and for quarterly data, 4 pairs of 2 workers were used." For the results, did the times reported in the paper represent running the ESRNN-GPU implementation with multiple workers in aggregate (CPU Time), multiple workers concurrently (Wall Clock Time) or was the time reported for a single worker?

    2. What GPU did you test on? Testing the M4 data set with CUDA enabled PyTorch on a notebook graphics cards (Nvidia GeForce GTX 1050) vs non-CUDA enabled PyTorch showed the CPU only version to be faster (i7 8550) by about 3x. It is likely that the PyTorch CPU enabled version is still faster the the original ESRNN, but I have not confirmed that.

    3. Is there any plan to implemented the future work for Variable Length Series mentioned in Section 8.1? What would be required?

    opened by tjphilpot 2
  • issue in upacking the project

    issue in upacking the project

    i have a problem in installing i type pip install git+https://github.com/damitkwr/ESRNN-GPU.git then error occurs then i type pip install git+https://github.com/damitkwr/ESRNN-GPU.git#egg=ESRNN-GPU then also error show plz help

    opened by 12suyash 2
  • ModuleNotFoundError: No module named 'ESRNN.m4_data'; 'ESRNN' is not a package

    ModuleNotFoundError: No module named 'ESRNN.m4_data'; 'ESRNN' is not a package

    Hello,

    I've tried to install ESRNN via the instruction in this link: https://pypi.org/project/ESRNN/ which were: pip install ESRNN

    However, when I try to run the follow code:

    from ESRNN.m4_data import prepare_m4_data from ESRNN.utils_evaluation import evaluate_prediction_owa

    from ESRNN import ESRNN

    I get the following error:

    Traceback (most recent call last): File "ESRNN.py", line 2, in from ESRNN.m4_data import prepare_m4_data File "C:\Users\mario\Documents\Python Benjamin\ESRNN.py", line 2, in from ESRNN.m4_data import prepare_m4_data ModuleNotFoundError: No module named 'ESRNN.m4_data'; 'ESRNN' is not a package

    I've tried adding ESRNN as a path variable and still get the same error.

    Could anyone please assist?

    opened by frenchmatthew 1
  • What is info.csv in es-rnn(main.py)?

    What is info.csv in es-rnn(main.py)?

    Hi, I am trying to use the code, but I encounter several issues. One is that, "train_path = '../data/Train/%s-train.csv' % (config['variable']) test_path = '../data/Test/%s-test.csv' % (config['variable'])" in main.py cause some problems. If I understand the code correctly, it changes the path to "../data/Train/Daily-train.csv", which is a CSV file that does not exist. The other issue is that I do not really understand what kind of information info.csv should contain. Would you please help me deal with these problems? Thanks

    opened by donovanfan 1
  • Epoch Loss not getting updated properly?

    Epoch Loss not getting updated properly?

    Edit: Never mind. The loss is just getting averaged as batch_num is outside of the for-loop that it increases in.

    In trainer.py inside the train method after an epoch finishes, the epoch_loss is divided by the batch_num + 1. That means that after every batch, the epoch_loss is forcefully decreased as the denominatior (the batch_num) is constantly getting bigger:

    epoch_loss = epoch_loss / (batch_num + 1)

    Maybe I'm misunderstanding something here, but it doesn't seem right that the loss is getting artificially decreased simply based on which batch the training loop is on. I looked through the original C++ implementation, but couldn't find anything that looked like the above line (I don't know C++ very well, so that may be why).

    P.S. Thanks for the python/torch implementation of this project btw, it's a great resource for learning some good forecasting methods/strategies.

    opened by waydegg 0
  • Train and test csv formatting

    Train and test csv formatting

    Hey, thank you for publishing your results, very impressive. What is the csv formatting for Train and Test? I've noticed that read_file creates arrays of different shapes: Train ends up in shape (number_of_series, ) Test - (number_of_series, time_steps)

    I would like to reproduce it on my data. How to format Train csv with pd.to_csv to be properly processed by your code?

    Thanks! Best regards

    opened by murometz 0
  • Fix seasonality extension.

    Fix seasonality extension.

    This can append too many months to the seasonalities. In the case of 'Monthly' data, config['output_size'] is 18, config['seasonality'] is 12. If seasonalities_stacked.shape[1] is 72, start_seasonality_ext is 60, so line 89 appends 12 seasonalities, rather than 6 (as in the C++ dynet code).

     //if prediction horizon is larger than seasonality, so we need to repeat some of the seasonality factors
                    if (OUTPUT_SIZE_I > SEASONALITY) {
                        unsigned long startSeasonalityIndx = season_exVect.size() - SEASONALITY;
                        for (int i = 0; i < (OUTPUT_SIZE_I - SEASONALITY); i++)
                            season_exVect.push_back(season_exVect[startSeasonalityIndx + i]);
                    }
    

    This becomes a problem on line 126 where we select the last config['output_size'], which will select the wrong output_size elements. In the case above, the seasonalities will be shifted by 6 months (18 - 12 = 6).

    opened by tjphilpot 0
  • showing syntax error on loading dataset...i have loaded the m4 dataset plz tell where in syntax i am wrong

    showing syntax error on loading dataset...i have loaded the m4 dataset plz tell where in syntax i am wrong

    def read_file(C:\Users\welcome\Downloads\m4 forecast\M4-methods-master\Dataset()):

    SyntaxError: unexpected character after line continuation character

    opened by 12suyash 0
  • Dataset is so complicated

    Dataset is so complicated

    Hi, I've tried to understand the dataset and how do you really train the model on it, but it seems that the information is not available about the competition anymore. Can you explain how the data is loaded for example from monthly Train.csv?

    opened by maxmax1992 0
  • The prediction intervals code

    The prediction intervals code

    Sorry to bother you. I can't find which part in this code implement the prediction intervals(PI) function. Is this code just for point forecast(PF)? Thank you.

    opened by HWZX 1
  • will it work for multivariate time series prediction   both regression and classification

    will it work for multivariate time series prediction both regression and classification

    great code thanks may you clarify : will it work for multivariate time series prediction both regression and classification 1 where all values are continues values weight height age target 1 56 160 34 1.2 2 77 170 54 3.5 3 87 167 43 0.7 4 55 198 72 0.5 5 88 176 32 2.3

    2 or even will it work for multivariate time series where values are mixture of continues and categorical values for example 2 dimensions have continues values and 3 dimensions are categorical values

    color        weight     gender  height  age  target 
    

    1 black 56 m 160 34 yes 2 white 77 f 170 54 no 3 yellow 87 m 167 43 yes 4 white 55 m 198 72 no 5 white 88 f 176 32 yes

    opened by Sandy4321 0
  • About the Input and output

    About the Input and output

    Hi there, My dataset only have two column- Date and Price. Is it possible to let the input be the price and the output be the price as well. If so, how should I divide them into x_train,y_train, and x_test, y_test. Will the algorithm do it for me automatically? Or if I only have these two columns, I will not be able to use this algorithm?

    opened by chendiva 0
Owner
Kaung
ML @google
Kaung
tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai

timeseriesAI 2.8k Jan 8, 2023
Time Series Forecasting with Temporal Fusion Transformer in Pytorch

Forecasting with the Temporal Fusion Transformer Multi-horizon forecasting often contains a complex mix of inputs – including static (i.e. time-invari

Nicolás Fornasari 6 Jan 24, 2022
Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting.

Non-AR Spatial-Temporal Transformer Introduction Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series For

Chen Kai 66 Nov 28, 2022
Multiple types of NN model optimization environments. It is possible to directly access the host PC GUI and the camera to verify the operation. Intel iHD GPU (iGPU) support. NVIDIA GPU (dGPU) support.

mtomo Multiple types of NN model optimization environments. It is possible to directly access the host PC GUI and the camera to verify the operation.

Katsuya Hyodo 24 Mar 2, 2022
Pytorch implementation of the popular Improv RNN model originally proposed by the Magenta team.

Pytorch Implementation of Improv RNN Overview This code is a pytorch implementation of the popular Improv RNN model originally implemented by the Mage

Sebastian Murgul 3 Nov 11, 2022
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting This is the origin Pytorch implementation of Informer in the followin

Haoyi 3.1k Dec 29, 2022
Spectral Temporal Graph Neural Network (StemGNN in short) for Multivariate Time-series Forecasting

Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting This repository is the official implementation of Spectral Temporal Gr

Microsoft 306 Dec 29, 2022
This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

TSForecasting This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the tim

Rakshitha Godahewa 80 Dec 30, 2022
Code for the CIKM 2019 paper "DSANet: Dual Self-Attention Network for Multivariate Time Series Forecasting".

Dual Self-Attention Network for Multivariate Time Series Forecasting 20.10.26 Update: Due to the difficulty of installation and code maintenance cause

Kyon Huang 223 Dec 16, 2022
The GitHub repository for the paper: “Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction“.

SCINet This is the original PyTorch implementation of the following work: Time Series is a Special Sequence: Forecasting with Sample Convolution and I

null 386 Jan 1, 2023
The source code and data of the paper "Instance-wise Graph-based Framework for Multivariate Time Series Forecasting".

IGMTF The source code and data of the paper "Instance-wise Graph-based Framework for Multivariate Time Series Forecasting". Requirements The framework

Wentao Xu 24 Dec 5, 2022
A pytorch implementation of Pytorch-Sketch-RNN

Pytorch-Sketch-RNN A pytorch implementation of https://arxiv.org/abs/1704.03477 In order to draw other things than cats, you will find more drawing da

Alexis David Jacq 172 Dec 12, 2022
LONG-TERM SERIES FORECASTING WITH QUERYSELECTOR – EFFICIENT MODEL OF SPARSEATTENTION

Query Selector Here you can find code and data loaders for the paper https://arxiv.org/pdf/2107.08687v1.pdf . Query Selector is a novel approach to sp

MORAI 62 Dec 17, 2022
Event-forecasting - Event Forecasting Algorithms With Python

event-forecasting Event Forecasting Algorithms Theory Correlating events in comp

Intellia ICT 4 Feb 15, 2022
Forecasting for knowable future events using Bayesian informative priors (forecasting with judgmental-adjustment).

What is judgyprophet? judgyprophet is a Bayesian forecasting algorithm based on Prophet, that enables forecasting while using information known by the

AstraZeneca 56 Oct 26, 2022
High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.

Anakin2.0 Welcome to the Anakin GitHub. Anakin is a cross-platform, high-performance inference engine, which is originally developed by Baidu engineer

null 514 Dec 28, 2022
GrabGpu_py: a scripts for grab gpu when gpu is free

GrabGpu_py a scripts for grab gpu when gpu is free. WaitCondition: gpu_memory >

tianyuluan 3 Jun 18, 2022
Static Features Classifier - A static features classifier for Point-Could clusters using an Attention-RNN model

Static Features Classifier This is a static features classifier for Point-Could

ABDALKARIM MOHTASIB 1 Jan 25, 2022
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Autoformer (NeurIPS 2021) Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting Time series forecasting is a c

THUML @ Tsinghua University 847 Jan 8, 2023