darts is a Python library for easy manipulation and forecasting of time series.

Overview

Time Series Made Easy in Python

darts


PyPI version GitHub Workflow Status Supported versions Docker Image Version (latest by date) GitHub Release Date Downloads Downloads

darts is a Python library for easy manipulation and forecasting of time series. It contains a variety of models, from classics such as ARIMA to deep neural networks. The models can all be used in the same way, using fit() and predict() functions, similar to scikit-learn. The library also makes it easy to backtest models, and combine the predictions of several models and external regressors. Darts supports both univariate and multivariate time series and models, and the neural networks can be trained on multiple time series.

Documentation

High Level Introductions

Install

We recommend to first setup a clean Python environment for your project with at least Python 3.7 using your favorite tool (conda, venv, virtualenv with or without virtualenvwrapper).

Once your environment is set up you can install darts using pip:

pip install darts

For more detailed install instructions you can refer to our installation guide at the end of this page.

Example Usage

Create a TimeSeries object from a Pandas DataFrame, and split it in train/validation series:

import pandas as pd
from darts import TimeSeries
df = pd.read_csv('AirPassengers.csv', delimiter=",")
series = TimeSeries.from_dataframe(df, 'Month', '#Passengers')
train, val = series.split_after(pd.Timestamp('19580101'))

Fit an exponential smoothing model, and make a prediction over the validation series' duration:

from darts.models import ExponentialSmoothing

model = ExponentialSmoothing()
model.fit(train)
prediction = model.predict(len(val))

Plot:

import matplotlib.pyplot as plt

series.plot(label='actual')
prediction.plot(label='forecast', lw=2)
plt.legend()
plt.xlabel('Year')
darts forecast example

We invite you to go over the example and tutorial notebooks in the examples directory.

Features

Currently, the library contains the following features:

Forecasting Models:

  • Exponential smoothing,
  • ARIMA & auto-ARIMA,
  • Facebook Prophet,
  • Theta method,
  • FFT (Fast Fourier Transform),
  • Recurrent neural networks (vanilla RNNs, GRU, and LSTM variants),
  • Temporal convolutional network.
  • Transformer
  • N-BEATS

Data processing: Tools to easily apply (and revert) common transformations on time series data (scaling, boxcox, …)

Metrics: A variety of metrics for evaluating time series' goodness of fit; from R2-scores to Mean Absolute Scaled Error.

Backtesting: Utilities for simulating historical forecasts, using moving time windows.

Regressive Models: Possibility to predict a time series from several other time series (e.g., external regressors), using arbitrary regressive models

Multivariate Support: Tools to create, manipulate and forecast multivariate time series.

Contribute

The development is ongoing, and there are many new features that we want to add. We welcome pull requests and issues on GitHub.

Before working on a contribution (a new feature or a fix), check our contribution guidelines.

Contact Us

If what you want to tell us is not a suitable github issue, feel free to send us an email at [email protected] for darts related matters or [email protected] for any other inquiries.

Installation Guide

Preconditions

Some of the models depend on fbprophet and torch, which have non-Python dependencies. A Conda environment is thus recommended because it will handle all of those in one go.

The following steps assume running inside a conda environment. If that's not possible, first follow the official instructions to install fbprophet and torch, then skip to Install darts

To create a conda environment for Python 3.7 (after installing conda):

conda create --name  python=3.7

Don't forget to activate your virtual environment

conda activate 

MAC

conda install -c conda-forge -c pytorch pip fbprophet pytorch

Linux and Windows

conda install -c conda-forge -c pytorch pip fbprophet pytorch cpuonly

Install darts

Install Darts with all available models: pip install darts.

As some models have relatively heavy (or non-Python) dependencies, we also maintain the u8darts package, which provides the following alternate lighter install options:

  • Install core only (without neural networks, Prophet or AutoARIMA): pip install u8darts
  • Install core + neural networks (PyTorch): pip install 'u8darts[torch]'
  • Install core + Facebook Prophet: pip install 'u8darts[fbprophet]'
  • Install core + AutoARIMA: pip install 'u8darts[pmdarima]'

Running the examples only, without installing:

If the conda setup is causing too many problems, we also provide a Docker image with everything set up for you and ready-to-use Python notebooks with demo examples. To run the example notebooks without installing our libraries natively on your machine, you can use our Docker image:

./gradlew docker && ./gradlew dockerRun

Then copy and paste the URL provided by the docker container into your browser to access Jupyter notebook.

For this setup to work you need to have a Docker service installed. You can get it at Docker website.

Tests

The gradle setup works best when used in a python environment, but the only requirement is to have pip installed for Python 3+

To run all tests at once just run

./gradlew test_all

alternatively you can run

./gradlew unitTest_all # to run only unittests
./gradlew coverageTest # to run coverage
./gradlew lint         # to run linter

To run the tests for specific flavours of the library, replace _all with _core, _fbprophet, _pmdarima or _torch.

Documentation

To build documantation locally just run

./gradlew buildDocs

After that docs will be available in ./docs/build/html directory. You can just open ./docs/build/html/index.html using your favourite browser.

Comments
  • [BUG]: 'numpy.ndarray' object has no attribute 'get_color' while plotting NaiveSessional() forecast

    [BUG]: 'numpy.ndarray' object has no attribute 'get_color' while plotting NaiveSessional() forecast

    Describe the bug A clear and concise description of what the bug is.

    While plotting a naive forecast an attributeError occured: 'numpy.ndarray object has no attribute get_color'

    To Reproduce Steps to reproduce the behavior, preferably code snippet. I have a subset of len 6188 'train' on which I trained the darts.NaiveSessional model.

    naive_model = models.NaiveSeasonal(K=1) naive_model.fit(train) naive_forecast = naive_model.predict(1) naive_forecast.plot(label='Forecast')


    AttributeError Traceback (most recent call last) c:\Users\Itcomplex\Documents\scripts\Predictive Analytics\Work\crypto_value_prediction_hourly.ipynb Cell 25' in <cell line: 4>() 2 naive_model.fit(train) 3 naive_forecast = naive_model.predict(1) ----> 4 naive_forecast.plot(label='Forecast')

    File c:\Users\Itcomplex\anaconda3\envs\env\lib\site-packages\darts\timeseries.py:2450, in TimeSeries.plot(self, new_plot, central_quantile, low_quantile, high_quantile, *args, **kwargs) 2447 kwargs["label"] = label_to_use 2449 p = central_series.plot(*args, **kwargs) -> 2450 color_used = p[0].get_color() 2451 kwargs["alpha"] = alpha if alpha is not None else alpha_confidence_intvls 2453 # Optionally show confidence intervals

    AttributeError: 'numpy.ndarray' object has no attribute 'get_color'

    Expected behavior A clear and concise description of what you expected to happen.

    Should have plotted the predicted timeseries.

    System (please complete the following information):

    • Python version: [e.g. 3.9.2]
    • darts version [e.g. 0.14.0]

    Additional context Add any other context about the problem here.

    bug 
    opened by iamMaverick 16
  • [BUG] Training never starts on TFT

    [BUG] Training never starts on TFT

    Describe the bug I use a dataset composed of 20 features and a single target. All of the features are future covariates. I use target past as well as the features's history as past covariates. To covariates, I add datetime attributes of year, month, day of week, hour, and holidays. The dataset has several years of hourly data, however I tried cutting down the samples to check if it made a difference. I am succesfully using the same dataset on other models (not from DARTS) and getting good results.

    To Reproduce

    train_ratio = 0.90
    look_back = 192
    horizon = 192
    n_outputs = 1
    
    df = pd.read_csv(file_path, index_col = 0)
    
    training_cutoff = pd.Timestamp(df['Time'].iloc[round(len(df)*train_ratio)])
    series = TimeSeries.from_dataframe(df, 'Time', value_cols = df.columns[1:])
    train, val = series.split_after(training_cutoff)
    
    scaler = Scaler()
    train_transformed = scaler.fit_transform(train)
    val_transformed = scaler.transform(val)
    series_transformed = scaler.transform(series)
    
    trgt_scaler = Scaler()
    trgt_transformed = trgt_scaler.fit_transform(series['target'])
    
    covariates = datetime_attribute_timeseries(series, attribute='year', one_hot=False)
    covariates = covariates.stack(datetime_attribute_timeseries(series, attribute='month', one_hot=False))
    covariates = covariates.stack(datetime_attribute_timeseries(series, attribute='day_of_week', one_hot=False))
    covariates = covariates.stack(datetime_attribute_timeseries(series, attribute='hour', one_hot=False))
    covariates = covariates.add_holidays(country)
    f_covariates = covariates.stack(TimeSeries.from_times_and_values(times=series.time_index, 
                                                                   values=df.iloc[:, 1+n_outputs:].to_numpy(), 
                                                                   columns=series.columns[n_outputs:]))
    p_covariates = covariates.stack(TimeSeries.from_times_and_values(times=series.time_index, 
                                                                   values=df.iloc[:, 1:].to_numpy(), 
                                                                   columns=series.columns))
    
    scaler_f_covs = Scaler()
    f_cov_train, f_cov_val = f_covariates.split_after(training_cutoff)
    scaler_f_covs.fit(f_cov_train)
    f_covariates_transformed = scaler_f_covs.transform(f_covariates)
    
    scaler_p_covs = Scaler()
    p_cov_train, p_cov_val = p_covariates.split_after(training_cutoff)
    scaler_p_covs.fit(p_cov_train)
    p_covariates_transformed = scaler_p_covs.transform(p_covariates)
    
    quantiles = [
         0.1, 0.25, 0.5, 0.75, 0.9
    ]
    model = TFTModel(input_chunk_length=look_back,
                        output_chunk_length=horizon,
                        hidden_size=32,
                        lstm_layers=1,
                        full_attention = True,
                        dropout = 0.1,
                        num_attention_heads=4,
                        batch_size=32,
                        n_epochs=250,
                        add_relative_index=False,
                        add_encoders=None,
                        #likelihood=None,
                        #loss_fn=MSELoss(),
                        likelihood=QuantileRegression(quantiles=quantiles),  # QuantileRegression is set per default
                        force_reset=True,
                        pl_trainer_kwargs = {"accelerator": "gpu", "gpus": [0], 
                                             "enable_progress_bar" : True, "enable_model_summary" : True},
                        optimizer_cls = torch.optim.SGD,
                        optimizer_kwargs = {'lr':0.01})
    
    model.fit(train_transformed['target'],
             future_covariates=f_covariates_transformed,
             past_covariates=p_covariates_transformed)
    
    
    

    Expected behavior Training starts but it gets stuck. It never ends a single epoch.

    System:

    • Python version: [ 3.9]
    • darts version [ 0.17.0]
    bug triage 
    opened by strakehyr 16
  • Windows installation

    Windows installation

    Hello, when installing darts libary using conda as per instructions with python 3.7 i get the following error:

    
    PS C:\Users\XXXX> conda install -c conda-forge u8darts-all
    Collecting package metadata (current_repodata.json): done
    Solving environment: failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
    Collecting package metadata (repodata.json): done
    Solving environment: failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: -
    Found conflicts! Looking for incompatible packages.
    This can take several minutes.  Press CTRL-C to abort.
    failed
    
    UnsatisfiableError: The following specifications were found to be incompatible with each other:
    
    Output in format: Requested package -> Available versions
    

    Do you have any ideas why this is happening?

    I can install prophet on it's own but not the darts libary.

    Thanks,

    Adam

    opened by adschem 16
  • Feat/format isort, linting, and pyupgrade

    Feat/format isort, linting, and pyupgrade

    Fixes #739

    Summary

    Extending the pre-commit linting.

    • isort
    • end-of-file-fixer
    • trailing-whitespace
    • check-added-large-files
    • requirements-txt-fixer
    • mixed-line-ending
    • pyupgrade

    each addition had its own commit to see what changed

    TODO:

    • [x] Add .git-blame-ignore-revs file for commit #765
    opened by gdevos010 15
  • Proposal for preprocessing pipeline

    Proposal for preprocessing pipeline

    Follow up on our discussion on preprocessing.

    BaseTransformer stay the same as was shown - but of course please raise again any concerns/improvement. transform in Timeseries also - again please raise any concerns/improvement.

    New class Pipeline for managing transforms and validations. Pipeline is intended only for one object of timeseries - this way when we call inverse we are sure that state of transformer is for correct series. Pipeline can be traversed forward and maybe backward (sometimes transform may not have inverse we may have to be extra careful about it)

    Interface of pipeline:

    add_transformer - function that add transformer to pipeline, order of transformers are determined as an order that transformers were added,
    add_validator - function that adds validator for each step
    resolve - function that resolves pipeline (traverse path of transformations to the end either forward or backward
    resolve_n_steps - same as resolve but traverse n step
    set_data/get_data -setter/getter for data - getter would return data that we have at our current step of transformations
    

    I left out implementations details as I feel like they are more appropriate to do after proposal in some form will be accepted.

    Ping @TheMP @hrzn @endrjuskr @pennfranc @guillaumeraille @radujica @Droxef. I hope I included all interested.

    opened by Kostiiii 15
  • [BUG] Timeseries.from_dataframe() does not handle localized datetime64 index correctly anymore

    [BUG] Timeseries.from_dataframe() does not handle localized datetime64 index correctly anymore

    Thank you for keeping up the great work on darts!

    Describe the bug In 0.9 with xarray backend of Timeseries, localized datetimeindex is not handled correctly (anymore). Using the toy example from another issue, you can see that plain datetimeindex is converted datetime64 in Timeseries. Once localized to UTC, the index is converted to object in Timeseries leading to downstream problems in plot(), etc.

    To reproduce

    df = pd.DataFrame(data={'First': [0,1,2,3,4,4.5,4,3,2,1]})
    inds = [f'2021-01-{day}' for day in range(1, 11)]
    df.index = pd.to_datetime(inds)
    
    df_localized = df.copy()
    df_localized.index = df.index.tz_localize('UTC')
    
    ts = TimeSeries.from_dataframe(df)
    ts_localized = TimeSeries.from_dataframe(df_localized)
    

    Expected behaviour Localized datetimeindex is converted to datetime64 in Timeseries.

    System Python version: Python 3.9.5 darts install: pip install u8darts

    opened by schweima 14
  • window transformer

    window transformer

    In addition to "lag features" which we already have, it'd be nice to add "window features", specifying window characteristics and corresponding function(s) to apply to create features dynamically in regression models. For instance, it is often helpful to use the trailing mean and variance of the last N points as features. We could also imagine having a way to have fairly generic windows (e.g., "last month", "last week", "the N points starting N-k time steps ago", etc...

    improvement 
    opened by hrzn 12
  • Training with common covariate for multiple timeseries

    Training with common covariate for multiple timeseries

    Hello,

    I want to train a global forecasting model for energy production from several generators. The generators are all in roughly the same geographical area (loc1), so I will assume that they share climate feature (common_climate_covariate_loc1). So I want to fit 25 generator TimeSeries to the model, and include a common covariate TimeSeries.

    Will this be the right way to approach this problem?

    model_cov.fit(series=[gen1_loc1, gen2_loc1, ... , genX_loc1], past_covariates=[common_climate_covariate_loc1], verbose=True)

    where: gen1_loc1 is generator1 from location 1 and so on...

    and later on, if I want to train the same model with 25 other generators that are from a different area (loc2) and have different climate data, do I just repeat the process and fit the new data to the same model?

    model_cov.fit(series=[gen_loc2_1, gen_loc2_2, ... , gen_loc3_X], past_covariates=[common_climate_covariate_loc2], verbose=True)

    where: gen1_loc2 is generator1 from location 2 and so on...

    Will this be the correct procedure? I hope I made the problem clear, thanks in advance for response.

    opened by Sigvesor 12
  • [BUG] RuntimeError: All input tensors must be on the same device. Received cuda:0 and cpu

    [BUG] RuntimeError: All input tensors must be on the same device. Received cuda:0 and cpu

    Describe the bug Getting the 'RuntimeError: All input tensors must be on the same device. Received cuda:0 and cpu' when trying to run the example notebook 05-RNN-examples cell 7 line eval_model(my_model)

    To Reproduce
    eval_model(my_model)


    RuntimeError Traceback (most recent call last) in ----> 1 eval_model(my_model)

    in eval_model(model) 1 def eval_model(model): ----> 2 pred_series = model.predict(n=26, covariates=covariates) 3 plt.figure(figsize=(8,5)) 4 series_transformed.plot(label='actual') 5 pred_series.plot(label='forecast')

    ~/anaconda3/envs/dart-time-series/lib/python3.8/site-packages/darts/utils/torch.py in decorator(self, *args, **kwargs) 63 with fork_rng(): 64 manual_seed(self._random_instance.randint(0, high=MAX_TORCH_SEED_VALUE)) ---> 65 return decorated(self, *args, **kwargs) 66 return decorator

    ~/anaconda3/envs/dart-time-series/lib/python3.8/site-packages/darts/models/torch_forecasting_model.py in predict(self, n, series, covariates, batch_size, verbose, n_jobs, roll_size, num_samples) 518 dataset = SimpleInferenceDataset(series, covariates, n, self.input_chunk_length, self.output_chunk_length, 519 self.is_recurrent) --> 520 predictions = self.predict_from_dataset(n, dataset, verbose=verbose, batch_size=batch_size, n_jobs=n_jobs, 521 roll_size=roll_size, num_samples=num_samples) 522 return predictions[0] if called_with_single_series else predictions

    ~/anaconda3/envs/dart-time-series/lib/python3.8/site-packages/darts/models/torch_forecasting_model.py in predict_from_dataset(self, n, input_series_dataset, batch_size, verbose, n_jobs, roll_size, num_samples) 619 for i in range(num_samples): 620 if self.is_recurrent: --> 621 batch_prediction = self._predict_batch_recurrent_model(n, input_series, cov_future) 622 else: 623 batch_prediction = self._predict_batch_block_model(n, input_series, cov_future, roll_size)

    ~/anaconda3/envs/dart-time-series/lib/python3.8/site-packages/darts/models/torch_forecasting_model.py in _predict_batch_recurrent_model(self, n, input_series, cov_future) 706 # create new input to model from last prediction and current covariates, if available 707 new_input = ( --> 708 torch.cat([out[:, -1:, :], cov_future[:, prediction_length - 1:prediction_length, :]], dim=2) 709 if cov_future is not None else out[:, -1:, :] 710 )

    RuntimeError: All input tensors must be on the same device. Received cuda:0 and cpu

    Expected behavior No bug when calling predict

    System (please complete the following information):

    • Python version: [3.8.10]
    • darts version [0.9.1]
    • torch [1.8.1+cu102]

    Additional context Issue with cuda version 11.3 but same issue when installing cuda 10.2 in the virtualenv with a slightly different error message (RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking arugment for argument tensors in method wrapper__cat))

    bug triage 
    opened by chouteag 12
  • Natural gaps in timeseries observations

    Natural gaps in timeseries observations

    I have hourly energy observations taken during business days only. So 120 hrs per week, not 168. With Sat and Sun missing always and holidays as well. Seasonality is daily, weekly, yearly.

    Was trying to follow samples and use TimeSeries.from_dataframe with default settings. I got a lot of NaNs inserted into DateTimeIndex, that matches pandas.asfreq('H') behaviour. So with train/test split train, val = series.split_before(pd.Timestamp('20200101')) I receive

    len(data[:'20200101']), len(train), len(data['20200101':]), len(val)
    (11856, 17328, 5904, 9480)
    
    data['20200101':]['load']
    dt_iso
    2020-01-09 00:00:00     801.0410
    2020-01-09 01:00:00     790.4990
    2020-01-09 02:00:00     770.1160
    2020-01-09 03:00:00     770.8910
    2020-01-09 04:00:00     774.4680
                             ...    
    2021-01-29 19:00:00   1,026.1950
    2021-01-29 20:00:00   1,007.2650
    2021-01-29 21:00:00     990.8280
    2021-01-29 22:00:00     953.9190
    2021-01-29 23:00:00     904.5980
    Name: load, Length: 5904, dtype: float64
    
    val['load']
                              load
    date                          
    2020-01-01 00:00:00        nan
    2020-01-01 01:00:00        nan
    2020-01-01 02:00:00        nan
    2020-01-01 03:00:00        nan
    2020-01-01 04:00:00        nan
    ...                        ...
    2021-01-29 19:00:00 1,026.1950
    2021-01-29 20:00:00 1,007.2650
    2021-01-29 21:00:00   990.8280
    2021-01-29 22:00:00   953.9190
    2021-01-29 23:00:00   904.5980
    
    [9480 rows x 1 columns]
    Freq: H
    

    So one can see that data has exploded with NaNs.

    Obviously darts.utils.statistics.plot_acf() and a darts.utils.statistics.check_seasonality() do not work with NaNs. plot_acf() gets me a straight line at zero, where should be AR lags up until 192. check_seasonality() reports [2021-03-05 17:39:16,120] INFO | darts.utils.statistics | The ACF has no local maximum for m < max_lag = 24.

    If I supplyfill_missing_dates=False for the TimeSeries.from_dataframe(): series = TimeSeries.from_dataframe(data, time_col='date', value_cols=['load'], fill_missing_dates=False) I get Could not infer frequency. Are some dates missing? Try specifying 'fill_missing_dates=True'

    With freq='H' parameter to function above no luck also. The same loophole as with pandas DateTimeIndex with freq='H'

    In statsmodels Sarimax models I was able to overcome datetime freq warning by converting index to PeriodIndex which supports gaps. data.index = pd.DatetimeIndex(data.index).to_period('H')

    Please advise what are my options with Darts to be able to work with time series data with natural gaps?

    Thanks a lot in advance for the attention.

    P.S. Some pictures to illustrate time series and gaps Capture-1 Capture

    feature request triage 
    opened by rmk17 12
  • fix: bad file descriptor

    fix: bad file descriptor

    Fixes #768 .

    Summary

    When using Prophet under a multi-threading environment such as Dask, the error [Errno 9] Bad File Descriptor could happen. I can consistently reproduce this error while running forecasts with Dask. This implementation seems to fix the issue.

    I do not fully understand why this happens, because only when running Dask for a long time would this happen. My implementation is semantically equivalent to the original, but I guess avoiding to open devnull too many times can help.

    Other Information

    I am running forecast using Prophet for about 500_000 time series, all in 1 big Dask data frames (split into 91 smaller Pandas data frames). If I don't use Dask, and instead use joblib to parallelize the forecasting, then this error does not happen. I think because joblib creates a separate thread for every worker, whereas Dask can have multiple threads for a worker.

    opened by nlhkh 11
  • `historical_forecasts()` retrains with brand-new model instances

    `historical_forecasts()` retrains with brand-new model instances

    Is your feature request related to a current problem? Please describe. historical_forecasts() retrains the same model instance on the new training section instead of using a brand-new model instance.

    Additional context The example code below shows that historical_forecasts() gives the same results as calling the fit() method multiple times on the same model instance. However, I would expect the outputs of historical_forecasts() to be the same as fitting a brand-new model instance, with the same parameters, on each training section.

    from darts.datasets import AirPassengersDataset
    from darts.dataprocessing.transformers import Scaler
    from darts.models.forecasting.block_rnn_model import BlockRNNModel
    import pandas as pd
    
    # A function for instantiating a rnn model with the initial weights fixed
    def instantiate_BlockRNNModel():
        return BlockRNNModel(
            input_chunk_length=12,
            output_chunk_length=6,
            model='RNN',
            hidden_dim=10,
            n_rnn_layers=1,
            hidden_fc_sizes=None,
            dropout=0,
            n_epochs=5,
            batch_size=16,
            optimizer_kwargs={"lr": 1e-3},
            random_state=42,
            save_checkpoints=True
        )
    
    # Prepare the data
    series = AirPassengersDataset().load()
    transformer = Scaler()
    series_transformed = transformer.fit_transform(series)
    
    # 1. `historical_forecasts()`, it is a 3-pass historical_forecast
    model = instantiate_BlockRNNModel()
    historical_forecasts_result = model.historical_forecasts(
        series_transformed,
        start=len(series_transformed)-3*6,
        train_length=None,
        forecast_horizon=6,
        stride=6,
        retrain=True,
        verbose=False,
        last_points_only=False
    )
    historical_forecasts_result = [forecast.values() for forecast in historical_forecasts_result]
    
    # 2. As far as I know, `historical_forecasts()` is similar to calling `fit()` on the same model with 3 training sections
    train_set_1, val_set_1 = series_transformed.split_after(pd.Timestamp('19590601'))
    train_set_2, val_set_2 = series_transformed.split_after(pd.Timestamp('19591201'))
    train_set_3, val_set_3 = series_transformed.split_after(pd.Timestamp('19600601'))
    
    individual_forecasts_by_same_model = []
    same_model = instantiate_BlockRNNModel()
    same_model.fit(train_set_1, verbose=True)
    individual_forecasts_by_same_model.append(same_model.predict(n=6).values())
    same_model.fit(train_set_2, verbose=True)
    individual_forecasts_by_same_model.append(same_model.predict(n=6).values())
    same_model.fit(train_set_3, verbose=True)
    individual_forecasts_by_same_model.append(same_model.predict(n=6).values())
    
    # 3. Fitting three individual model instances with 3 training sections
    # From the previous two parts, `historical_forecasts_result` is the same as `individual_forecasts_by_same_model`
    # However, I would expect the results given by the `historical_forecasts()` to be the same as 
    # fitting a brand-new model on each training section, like the following `individual_forecasts_by_different_models`.
    train_set_1, val_set_1 = series_transformed.split_after(pd.Timestamp('19590601'))
    train_set_2, val_set_2 = series_transformed.split_after(pd.Timestamp('19591201'))
    train_set_3, val_set_3 = series_transformed.split_after(pd.Timestamp('19600601'))
    
    individual_model_1 = instantiate_BlockRNNModel()
    individual_model_2 = instantiate_BlockRNNModel()
    individual_model_3 = instantiate_BlockRNNModel()
    individual_model_1.fit(train_set_1, verbose=True)
    individual_model_2.fit(train_set_2, verbose=True)
    individual_model_3.fit(train_set_3, verbose=True)
    individual_forecasts_by_different_models = [
        model.predict(n=6).values() 
        for model
        in [individual_model_1, individual_model_2, individual_model_3]
    ]
    

    Result

    historical_forecasts_result:
    [array([[0.40726016],
            [0.14743934],
            [0.42281431],
            [0.30751979],
            [0.3935865 ],
            [0.23179494]]),
     array([[0.40049573],
            [0.38148109],
            [0.45629141],
            [0.3826234 ],
            [0.38148096],
            [0.40922707]]),
     array([[0.53207854],
            [0.50775845],
            [0.5872545 ],
            [0.43763365],
            [0.43169829],
            [0.55752873]])]
    
    individual_forecasts_by_same_model:
    [array([[0.40726016],
            [0.14743934],
            [0.42281431],
            [0.30751979],
            [0.3935865 ],
            [0.23179494]]),
     array([[0.40049573],
            [0.38148109],
            [0.45629141],
            [0.3826234 ],
            [0.38148096],
            [0.40922707]]),
     array([[0.53207854],
            [0.50775845],
            [0.5872545 ],
            [0.43763365],
            [0.43169829],
            [0.55752873]])]
    
    individual_forecasts_by_different_models
    [array([[0.40726016],
            [0.14743934],
            [0.42281431],
            [0.30751979],
            [0.3935865 ],
            [0.23179494]]),
     array([[0.402741  ],
            [0.15947664],
            [0.42768211],
            [0.34189958],
            [0.42966399],
            [0.25139858]]),
     array([[0.46655868],
            [0.2012079 ],
            [0.50961595],
            [0.35336985],
            [0.44060068],
            [0.28922872]])]
    

    Describe proposed solution Save the model before any training, and load the untrained model before retraining. Alternatively, reset the model to its initial weights before retraining.

    triage 
    opened by ericlikp 0
  • [BUG] Static covariates error when training global models on lists of TimeSeries

    [BUG] Static covariates error when training global models on lists of TimeSeries

    Describe the bug A ValueError is thrown when attempting to train global models (I've tried LightGBMModel and LinearRegressionModel) on lists of TimeSeries objects that all have the same static covariates assigned.

    The static covariate labels are strings and I've tried both with and without using StaticCovariatesTransformer() to convert the strings to float labels.

    To Reproduce All TimeSeries objects have the same structure but vary in length. For example: image

    AND

    image

    Fit the model:

    model = LightGBMModel(lags=[-1] , output_chunk_length=1)
    model.fit(all_train_scaled, future_covariates=None)
    

    Error:

    /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/darts/utils/data/tabularization.py in _add_static_covariates(model, series, features)
        219 
        220         # concatenate static covariates to features
    --> 221         return np.concatenate([features, static_covs], axis=1)
    
    /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/numpy/core/overrides.py in concatenate(*args, **kwargs)
    
    ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 16571 and the array at index 1 has size 16570
    

    Expected behavior The static_covs array should match the dimensions of the features array so that they can be concatenated.

    System (please complete the following information):

    • Python version: 3.9.5
    • darts version 0.23.0
    • lightgbm version 3.3.2

    Additional context DARTS has been a huge boost for me. Thanks for all the effort, especially the effort put into the docs.

    bug triage 
    opened by rgrosskopf 0
  • fix deep copying of TFM trainer parameters

    fix deep copying of TFM trainer parameters

    Fixes #1455.

    Summary

    • fixes mutable copying of trainer parameters in TorchForecastingModels. This resulted in PyTorch Lightning appending their callbacks to the original trainer_params.
    opened by dennisbader 1
  • [QUESTION] How does one deal with updating/republishing forecasts as historical future covariates

    [QUESTION] How does one deal with updating/republishing forecasts as historical future covariates

    Hi Darts Team,

    I understand that external forecasts can be used as a future and historical future covariates, but how does one incorporate the fact that forecasts are typically republished at a given frequency when making a backtest?

    E.g. following the example of River flow forecasting at https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993, let's say one wants to do a backtest applying rolling window predictions, and assume that there is a new rain weather forecast every day. How does one "use" the historical forecasts, such that the model will utilize the newest forecast at the prediction point, but not a forecast which hasn't been released yet at that time. Put in other words, is it somehow possible to state the publish time of a historical future covariate, and instruct Darts to only use the information available at that time when making predictions in a backtest.

    triage 
    opened by frederikgnie 0
Releases(0.23.0)
Owner
Unit8
Solving your most impactful problems via Big Data & AI
Unit8
Delorean: Time Travel Made Easy

Delorean: Time Travel Made Easy Delorean is a library for clearing up the inconvenient truths that arise dealing with datetimes in Python. Understandi

Mahdi Yusuf 1.8k Dec 20, 2022
⌚️Internet Time reference and (eventually) converter site, for planning things with your internet friends who aren't (yet) obsessed with Internet Time 😉

Internet-Ti.me Internet Time reference and (eventually) converter site, for planning things with your internet friends who aren't (yet) obsessed with

Jessica Stokes 17 Nov 2, 2022
UNIX time from NTP or short UtfN is a simple CLI tool to set the time from an NTP-Server.

UNIX ⌚ from NTP UNIX time from NTP or short UtfN is a simple CLI tool to set the time from an NTP-Server. Sets time and date using the date command pr

Alexander 1 Jan 2, 2022
Python datetimes made easy

Pendulum Python datetimes made easy. Supports Python 2.7 and 3.4+. >>> import pendulum >>> now_in_paris = pendulum.now('Europe/Paris') >>> now_in_par

Sébastien Eustace 5.3k Jan 6, 2023
A simple in-process python scheduler library, designed to be integrated seamlessly with the `datetime` standard library.

scheduler A simple in-process python scheduler library, designed to be integrated seamlessly with the datetime standard library. Due to the support of

null 30 Dec 30, 2022
TimeTagger is a web-based time-tracking solution that can be run locally or on a server

TimeTagger is a web-based time-tracking solution that can be run locally or on a server. In the latter case, you'll want to add authentication, and also be aware of the license restrictions.

Almar Klein 626 Jan 6, 2023
Parse human-readable date/time strings

parsedatetime Parse human-readable date/time strings. Python 2.6 or greater is required for parsedatetime version 1.0 or greater. While we still test

Mike Taylor 651 Dec 23, 2022
ISO 8601 date/time parser

ISO 8601 date/time parser This module implements ISO 8601 date, time and duration parsing. The implementation follows ISO8601:2004 standard, and imple

null 118 Dec 20, 2022
Cross Platform Application for Calculating Render Time

mdsanima-rt-go Cross Platform Application for Calculating Render Time. Testing This is a base application build on Windows Android and Linux. All buil

MDSANIMA DEV 2 Mar 29, 2022
A Python 3 library for parsing human-written times and dates

Chronyk A small Python 3 library containing some handy tools for handling time, especially when it comes to interfacing with those pesky humans. Featu

Felix Wiegand 339 Dec 19, 2022
pytz Python historical timezone library and database

pytz Brings the IANA tz database into Python. This library allows accurate and cross platform timezone calculations. pytz contains generated code, and

Stub 236 Jan 3, 2023
A Python library for dealing with dates

moment A Python library for dealing with dates/times. Inspired by Moment.js and Kenneth Reitz's Requests library. Ideas were also taken from the Times

Zach Williams 709 Dec 9, 2022
Generate and work with holidays in Python

python-holidays A fast, efficient Python library for generating country, province and state specific sets of holidays on the fly. It aims to make dete

Maurizio Montel 881 Dec 29, 2022
A datetime parser in Python by Ari24-cb24 and NekoFantic

datetimeparser A datetime parser in Python by Ari24-cb24 and NekoFantic V 1.0 Erinnerung für den Parser Auf falsche Eingaben überprüfen Liste an Event

AriDevelopment 13 Dec 30, 2022
Useful extensions to the standard Python datetime features

dateutil - powerful extensions to datetime The dateutil module provides powerful extensions to the standard datetime module, available in Python. Inst

null 2k Dec 29, 2022
Friendly Python Dates

When.py: Friendly Dates and Times Production: Development: User-friendly functions to help perform common date and time actions. Usage To get the syst

Andy Dirnberger 191 Oct 14, 2022
Better dates & times for Python

Arrow: Better dates & times for Python Arrow is a Python library that offers a sensible and human-friendly approach to creating, manipulating, formatt

Arrow 8.2k Jan 5, 2023
python parser for human readable dates

Python parser for human readable dates Key Features • How To Use • Installation • Common use cases • You may also like... • License Key Features Suppo

Scrapinghub 2.2k Jan 8, 2023
A Python module that tries to figure out what your local timezone is

tzlocal This Python module returns a tzinfo object with the local timezone information under Unix and Windows. It requires either Python 3.9+ or the b

Lennart Regebro 159 Dec 16, 2022