PyPOTS - A Python Toolbox for Data Mining on Partially-Observed Time Series

Overview

Welcome to PyPOTS

A Python Toolbox for Data Mining on Partially-Observed Time Series

PyPI

⦿ Motivation: Due to all kinds of reasons like failure of collection sensors, communication error, and unexpected malfunction, missing values are common to see in time series from the real-world environment. This makes partially-observed time series (POTS) a pervasive problem in open-world modeling and prevents advanced data analysis. Although this problem is important, the area of data mining on POTS still lacks a dedicated toolkit. PyPOTS is created to fill in this blank.

⦿ Mission: PyPOTS is born to become a handy toolbox that is going to make data mining on POTS easy rather than tedious, to help engineers and researchers focus more on the core problems in their hands rather than on how to deal with the missing parts in their data. PyPOTS will keep integrating classical and the latest state-of-the-art data mining algorithms for partially-observed multivariate time series. For sure, besides various algorithms, PyPOTS is going to have unified APIs together with detailed documentation and interactive examples across algorithms as tutorials.

To make various open-source time-series datasets readily available to our users, PyPOTS gets supported by project TSDB (Time-Series DataBase), a toolbox making loading time-series datasets super easy!

Visit TSDB right now to know more about this handy tool 🛠 !

❖ Installation

Install the latest release from PyPI:

pip install pypots

Install with the latest code on GitHub:

pip install https://github.com/WenjieDu/PyPOTS/archive/main.zip

❖ Available Algorithms

Task Type Algorithm Year Reference
Imputation Neural Network SAITS: Self-Attention-based Imputation for Time Series 2022 1
Imputation Neural Network Transformer 2017 2 1
Imputation,
Classification
Neural Network BRITS (Bidirectional Recurrent Imputation for Time Series) 2018 3
Imputation Naive LOCF (Last Observation Carried Forward) - -
Classification Neural Network GRU-D 2018 4
Classification Neural Network Raindrop 2022 5
Clustering Neural Network CRLI (Clustering Representation Learning on Incomplete time-series data) 2021 6
Clustering Neural Network VaDER (Variational Deep Embedding with Recurrence) 2019 7
Forecasting Probabilistic BTTF (Bayesian Temporal Tensor Factorization) 2021 8

‼️ PyPOTS is currently under developing. If you like it and look forward to its growth, please give PyPOTS a star and watch it to keep you posted on its progress and to let me know that its development is meaningful. If you have any feedback, or want to contribute ideas/suggestions or share time-series related algorithms/papers, please join PyPOTS community and , or drop me an email.

Thank you all for your attention! 😃

Footnotes

  1. Du, W., Cote, D., & Liu, Y. (2022). SAITS: Self-Attention-based Imputation for Time Series. ArXiv, abs/2202.08516. 2

  2. Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. NeurIPS 2017.

  3. Cao, W., Wang, D., Li, J., Zhou, H., Li, L., & Li, Y. (2018). BRITS: Bidirectional Recurrent Imputation for Time Series. NeurIPS 2018.

  4. Che, Z., Purushotham, S., Cho, K., Sontag, D.A., & Liu, Y. (2018). Recurrent Neural Networks for Multivariate Time Series with Missing Values. Scientific Reports, 8.

  5. Zhang, X., Zeman, M., Tsiligkaridis, T., & Zitnik, M. (2022). Graph-Guided Network for Irregularly Sampled Multivariate Time Series. ICLR 2022.

  6. Ma, Q., Chen, C., Li, S., & Cottrell, G. W. (2021). Learning Representations for Incomplete Time Series Clustering. AAAI 2021.

  7. Jong, J.D., Emon, M.A., Wu, P., Karki, R., Sood, M., Godard, P., Ahmad, A., Vrooman, H.A., Hofmann-Apitius, M., & Fröhlich, H. (2019). Deep learning for clustering of multivariate clinical patient trajectories with missing values. GigaScience, 8.

  8. Sun, L., & Chen, X. (2021). Bayesian Temporal Factorization for Multidimensional Time Series Prediction. IEEE transactions on pattern analysis and machine intelligence, PP.

Issues
  • GPU enabled model raises Exception: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0

    GPU enabled model raises Exception: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0

    Hello, great library, but using gpu enabled machine results in errors.

    pypots version = 0.0.6 (the one available in PyPI)

    code to replicate problem:

    import unittest
    from pypots.tests.test_imputation import TestBRITS, TestLOCF, TestSAITS, TestTransformer
    from pypots import __version__
    
    
    if __name__ == "__main__":
        print(__version__)
        unittest.main()
    

    results:

    0.0.6
    Running test cases for BRITS...
    Model initialized successfully. Number of the trainable parameters: 580976
    ERunning test cases for BRITS...
    Model initialized successfully. Number of the trainable parameters: 580976
    ERunning test cases for LOCF...
    LOCF test_MAE: 0.1712224306027283
    .Running test cases for LOCF...
    .Running test cases for SAITS...
    Model initialized successfully. Number of the trainable parameters: 1332704
    Exception: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0
    ERunning test cases for SAITS...
    Model initialized successfully. Number of the trainable parameters: 1332704
    Exception: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0
    ERunning test cases for Transformer...
    Model initialized successfully. Number of the trainable parameters: 666122
    epoch 0: training loss 0.7681, validating loss 0.2941
    epoch 1: training loss 0.4731, validating loss 0.2395
    epoch 2: training loss 0.4235, validating loss 0.2069
    epoch 3: training loss 0.3781, validating loss 0.1914
    epoch 4: training loss 0.3530, validating loss 0.1837
    ERunning test cases for Transformer...
    Model initialized successfully. Number of the trainable parameters: 666122
    epoch 0: training loss 0.7826, validating loss 0.2820
    epoch 1: training loss 0.4687, validating loss 0.2352
    epoch 2: training loss 0.4188, validating loss 0.2132
    epoch 3: training loss 0.3857, validating loss 0.1977
    epoch 4: training loss 0.3604, validating loss 0.1945
    E
    ======================================================================
    ERROR: test_impute (pypots.tests.test_imputation.TestBRITS)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 99, in setUp
        self.brits.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/brits.py", line 494, in fit
        training_set = DatasetForBRITS(train_X)  # time_gaps is necessary for BRITS
      File "mydirs(...)/python3.9/site-packages/pypots/data/dataset_for_brits.py", line 62, in __init__
        forward_delta = parse_delta(forward_missing_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/data/dataset_for_brits.py", line 36, in parse_delta
        delta.append(torch.ones(1, n_features) + (1 - m_mask[step]) * delta[-1])
    RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
    
    ======================================================================
    ERROR: test_parameters (pypots.tests.test_imputation.TestBRITS)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 99, in setUp
        self.brits.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/brits.py", line 494, in fit
        training_set = DatasetForBRITS(train_X)  # time_gaps is necessary for BRITS
      File "mydirs(...)/python3.9/site-packages/pypots/data/dataset_for_brits.py", line 62, in __init__
        forward_delta = parse_delta(forward_missing_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/data/dataset_for_brits.py", line 36, in parse_delta
        delta.append(torch.ones(1, n_features) + (1 - m_mask[step]) * delta[-1])
    RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
    
    ======================================================================
    ERROR: test_impute (pypots.tests.test_imputation.TestSAITS)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 83, in _train_model
        results = self.model.forward(inputs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 95, in forward
        imputed_data, [X_tilde_1, X_tilde_2, X_tilde_3] = self.impute(inputs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 62, in impute
        enc_output, _ = encoder_layer(enc_output)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 122, in forward
        enc_output, attn_weights = self.slf_attn(enc_input, enc_input, enc_input, attn_mask=mask_time)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 72, in forward
        v, attn_weights = self.attention(q, k, v, attn_mask)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 32, in forward
        attn = attn.masked_fill(attn_mask == 1, -1e9)
    RuntimeError: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 35, in setUp
        self.saits.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 171, in fit
        self._train_model(training_loader, val_loader, val_X_intact, val_X_indicating_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 123, in _train_model
        raise RuntimeError('Training got interrupted. Model was not get trained. Please try fit() again.')
    RuntimeError: Training got interrupted. Model was not get trained. Please try fit() again.
    
    ======================================================================
    ERROR: test_parameters (pypots.tests.test_imputation.TestSAITS)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 83, in _train_model
        results = self.model.forward(inputs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 95, in forward
        imputed_data, [X_tilde_1, X_tilde_2, X_tilde_3] = self.impute(inputs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 62, in impute
        enc_output, _ = encoder_layer(enc_output)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 122, in forward
        enc_output, attn_weights = self.slf_attn(enc_input, enc_input, enc_input, attn_mask=mask_time)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 72, in forward
        v, attn_weights = self.attention(q, k, v, attn_mask)
      File "mydirs(...)/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
        return forward_call(*input, **kwargs)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 32, in forward
        attn = attn.masked_fill(attn_mask == 1, -1e9)
    RuntimeError: expected self and mask to be on the same device, but got mask on cpu and self on cuda:0
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 35, in setUp
        self.saits.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/saits.py", line 171, in fit
        self._train_model(training_loader, val_loader, val_X_intact, val_X_indicating_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 123, in _train_model
        raise RuntimeError('Training got interrupted. Model was not get trained. Please try fit() again.')
    RuntimeError: Training got interrupted. Model was not get trained. Please try fit() again.
    
    ======================================================================
    ERROR: test_impute (pypots.tests.test_imputation.TestTransformer)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 68, in setUp
        self.transformer.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 257, in fit
        self._train_model(training_loader, val_loader, val_X_intact, val_X_indicating_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 129, in _train_model
        if np.equal(self.best_loss, float('inf')):
      File "mydirs(...)/python3.9/site-packages/torch/_tensor.py", line 732, in __array__
        return self.numpy()
    TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
    
    ======================================================================
    ERROR: test_parameters (pypots.tests.test_imputation.TestTransformer)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "mydirs(...)/python3.9/site-packages/pypots/tests/test_imputation.py", line 68, in setUp
        self.transformer.fit(self.train_X, self.val_X)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/transformer.py", line 257, in fit
        self._train_model(training_loader, val_loader, val_X_intact, val_X_indicating_mask)
      File "mydirs(...)/python3.9/site-packages/pypots/imputation/base.py", line 129, in _train_model
        if np.equal(self.best_loss, float('inf')):
      File "mydirs(...)/python3.9/site-packages/torch/_tensor.py", line 732, in __array__
        return self.numpy()
    TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
    
    ----------------------------------------------------------------------
    Ran 8 tests in 20.239s
    
    FAILED (errors=6)
    

    i suspect that you call .to(device) too early on data. You might also override device parameter when initiating new tensors (i.e. in torch.ones in parse_delta)

    Best regards!

    opened by MaciejSkrabski 4
  • Early stop

    Early stop

    Wenjie,

    I tried the PyPOTS with the Beijing Air quality database. For the dataset preparation, I follow the gene_UCI_BeijingAirQuality_dataset. The following is the PyPOTS setup.

    saits_base = SAITS(seq_len=seq_len, n_features=132, 
                       n_layers=2,  # num of group-inner layers
                       d_model=256, # model hidden dim
                       d_inner=128, # hidden size of feed forward layer
                       n_head=4, # head num of self-attention
                       d_k=64, d_v=64, # key dim, value dim
                       dropout=0, 
                       epochs=200,
                       patience=30,
                       batch_size=32,
                       weight_decay=1e-5,
                       ORT_weight=1,
                       MIT_weight=1,
                      )
    
    saits_base.fit(train_set_X)
    

    PyPOTS stops earlier than the epochs specified (stops around epoch 80), without triggering either print('Exceeded the training patience. Terminating the training procedure...') or print('Finished all training epochs.').

    epoch 0: training loss 0.9637 
    epoch 1: training loss 0.6161 
    epoch 2: training loss 0.5177 
    epoch 3: training loss 0.4783 
    epoch 4: training loss 0.4489 
    ...
    epoch 73: training loss 0.2462 
    epoch 74: training loss 0.2460 
    epoch 75: training loss 0.2480 
    epoch 76: training loss 0.2452 
    epoch 77: training loss 0.2452 
    epoch 78: training loss 0.2458 
    epoch 79: training loss 0.2449 
    epoch 80: training loss 0.2423 
    epoch 81: training loss 0.2425 
    epoch 82: training loss 0.2443 
    epoch 83: training loss 0.2403 
    epoch 84: training loss 0.2406
    
    

    Then I evaluate the model performance (not knowing why the model stops early) on test_set as

    test_set_mae = cal_mae(test_set_imputation, test_set_X_intact, test_set_indicating_mask)
    0.21866121846582318
    

    I have a few questions:

    1. What could be the cause for the early stop?
    2. In addition, is there any object in saits_base that stores the loss history?
    3. Does the function cal_mae calculate the same MAE in your paper? For this Beijing air quality case, I should be able to tune the hyperparameter to get the test_set_mae down to around 0.146?

    Thank you, Haochen

    opened by Rdfing 2
  • fix: brits on cuda

    fix: brits on cuda

    Some tensors created on the fly (mainly in base.py and dataset_for_brits.py ) used to ignore the model's and data's device (cpu or gpu). This caused BRITS to throw errors whenever users wanted to run it on cuda enabled machine.

    opened by MaciejSkrabski 1
  • BRITS imputation test fails on cuda device mismatch

    BRITS imputation test fails on cuda device mismatch

    Hi, when trying to run imputation tests with commit 6dcc8942459094e3a3fc5e11363f5d712ee8e742 on dev branch.

    py3.9_cuda11.3_cudnn8.2.0_0

    $ python -m pytest tests/test_imputation.py
    
    ./tests/test_imputation.py::TestBRITS::test_parameters Failed with Error: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
      File ".../unittest/case.py", line 59, in testPartExecutor
        yield
      File ".../unittest/case.py", line 588, in run
        self._callSetUp()
      File ".../unittest/case.py", line 547, in _callSetUp
        self.setUp()
      File ".../PyPOTS/pypots/tests/test_imputation.py", line 98, in setUp
        self.brits.fit(self.train_X, self.val_X)
      File "/PyPOTS/pypots/imputation/brits.py", line 504, in fit
        self._train_model(training_loader, val_loader, val_X_intact, val_X_indicating_mask)
      File "/PyPOTS/pypots/imputation/base.py", line 154, in _train_model
        if np.equal(self.best_loss, float("inf")):
      File .../lib/python3.9/site-packages/torch/_tensor.py", line 732, in __array__
        return self.numpy()
    TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
    
    opened by MaciejSkrabski 3
  • refactor: explicit channels in conda env ymls

    refactor: explicit channels in conda env ymls

    Hi! You may have noticed that, when creating a new conda environment from *.yml file, it takes ages to solve package dependencies. I attempt to speed the process up by explicitly defining channel in which to search for a package. I also defined minimal pandas version to be 1.4.1 - the things were weird before that. I also allow for python versions newer than 3.7.13 and I believe you'll find it acceptable.

    Please let me know if this is in any way helpful.

    opened by MaciejSkrabski 9
Releases(v0.0.7)
Owner
Wenjie Du
"Do one thing, and do it well."
Wenjie Du
Kats is a toolkit to analyze time series data, a lightweight, easy-to-use, and generalizable framework to perform time series analysis.

Kats, a kit to analyze time series data, a lightweight, easy-to-use, generalizable, and extendable framework to perform time series analysis, from understanding the key statistics and characteristics, detecting change points and anomalies, to forecasting future trends.

Facebook Research 3.9k Aug 5, 2022
A data preprocessing package for time series data. Design for machine learning and deep learning.

A data preprocessing package for time series data. Design for machine learning and deep learning.

Allen Chiang 144 Aug 5, 2022
K-means clustering is a method used for clustering analysis, especially in data mining and statistics.

K Means Algorithm What is K Means This algorithm is an iterative algorithm that partitions the dataset according to their features into K number of pr

null 1 Nov 1, 2021
A machine learning toolkit dedicated to time-series data

tslearn The machine learning toolkit for time series analysis in Python Section Description Installation Installing the dependencies and tslearn Getti

null 2.2k Aug 13, 2022
Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.

Prophet: Automatic Forecasting Procedure Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends ar

Facebook 14.8k Aug 9, 2022
A machine learning toolkit dedicated to time-series data

tslearn The machine learning toolkit for time series analysis in Python Section Description Installation Installing the dependencies and tslearn Getti

null 2.2k Aug 9, 2022
Automatically build ARIMA, SARIMAX, VAR, FB Prophet and XGBoost Models on Time Series data sets with a Single Line of Code. Now updated with Dask to handle millions of rows.

Auto_TS: Auto_TimeSeries Automatically build multiple Time Series models using a Single Line of Code. Now updated with Dask. Auto_timeseries is a comp

AutoViz and Auto_ViML 466 Aug 8, 2022
Visualize classified time series data with interactive Sankey plots in Google Earth Engine

sankee Visualize changes in classified time series data with interactive Sankey plots in Google Earth Engine Contents Description Installation Using P

Aaron Zuspan 73 Jul 30, 2022
MaD GUI is a basis for graphical annotation and computational analysis of time series data.

MaD GUI Machine Learning and Data Analytics Graphical User Interface MaD GUI is a basis for graphical annotation and computational analysis of time se

Machine Learning and Data Analytics Lab FAU 6 Aug 9, 2022
Open source time series library for Python

PyFlux PyFlux is an open source time series library for Python. The library has a good array of modern time series models, as well as a flexible array

Ross Taylor 2k Aug 7, 2022
A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.

pmdarima Pmdarima (originally pyramid-arima, for the anagram of 'py' + 'arima') is a statistical library designed to fill the void in Python's time se

alkaline-ml 1.2k Aug 15, 2022
Probabilistic time series modeling in Python

GluonTS - Probabilistic Time Series Modeling in Python GluonTS is a Python toolkit for probabilistic time series modeling, built around Apache MXNet (

Amazon Web Services - Labs 2.9k Aug 15, 2022
A python library for easy manipulation and forecasting of time series.

Time Series Made Easy in Python darts is a python library for easy manipulation and forecasting of time series. It contains a variety of models, from

Unit8 4.5k Aug 12, 2022
A Python package for time series classification

pyts: a Python package for time series classification pyts is a Python package for time series classification. It aims to make time series classificat

Johann Faouzi 1.3k Aug 9, 2022
Python module for machine learning time series:

seglearn Seglearn is a python package for machine learning time series or sequences. It provides an integrated pipeline for segmentation, feature extr

David Burns 522 Jul 31, 2022
A Python toolkit for rule-based/unsupervised anomaly detection in time series

Anomaly Detection Toolkit (ADTK) Anomaly Detection Toolkit (ADTK) is a Python package for unsupervised / rule-based time series anomaly detection. As

Arundo Analytics 844 Aug 10, 2022
AtsPy: Automated Time Series Models in Python (by @firmai)

Automated Time Series Models in Python (AtsPy) SSRN Report Easily develop state of the art time series models to forecast univariate data series. Simp

Derek Snow 451 Aug 5, 2022
A python library for Bayesian time series modeling

PyDLM Welcome to pydlm, a flexible time series modeling library for python. This library is based on the Bayesian dynamic linear model (Harrison and W

Sam 425 Aug 7, 2022
A Python implementation of GRAIL, a generic framework to learn compact time series representations.

GRAIL A Python implementation of GRAIL, a generic framework to learn compact time series representations. Requirements Python 3.6+ numpy scipy tslearn

null 3 Nov 24, 2021