Describe the bug After fitting the model, using model.predict(steps_ahead=n), it goes to 100% but then throws the following error: "unsupported operand type(s) for +: 'Timestamp' and 'NoneType' when using model.predict"

To Reproduce I have a dataframe very similarly set up to the M5 notebook. I can't see any things in how it would differ besides the numbers. The hierarchy is set up correctly.

Expected behavior A clear and concise description of what you expected to happen. model.predict() should finish without errors.

Desktop (please complete the following information):

• OS: OS X Catalina
• scikit-hts version: 0.5.1
• Python version: 3.7.6

Additional context I don't know exactly how predict works, so it seems it wants to use the index (datetime) and predict the next step..?

I want to revise my base forecasts, so wanted to use revise_forecasts() method but got an error like this.

• scikit-hts version: scikit-hts==0.5.4
• Python version: 3.6
• Operating System: Windows

AttributeError                            Traceback (most recent call last)
<ipython-input-191-307987f74b26> in <module>
----> 1 revise_forecasts(method = 'FP', forecasts = pred, summing_matrix = sum_mat, nodes = hierarchyNodes)

/home/Gopal/anaconda/envs/time_series/lib/python3.6/site-packages/hts/convenience.py in revise_forecasts(method, forecasts, errors, residuals, summing_matrix, nodes, transformer)
     63     )
     64 
---> 65     revised = revision.revise(forecasts=forecasts, mse=errors, nodes=nodes)
     66 
     67     return pandas.DataFrame(revised, columns=list(forecasts.keys()))

/home/Gopal/anaconda/envs/time_series/lib/python3.6/site-packages/hts/revision.py in revise(self, forecasts, mse, nodes)
     74 
     75         elif self.name == MethodT.FP.name:
---> 76             return forecast_proportions(forecasts, nodes)
     77 
     78         else:

/home/Gopal/anaconda/envs/time_series/lib/python3.6/site-packages/hts/functions.py in forecast_proportions(forecasts, nodes)
    196 
    197     key = choice(list(forecasts.keys()))
--> 198     new_mat = np.empty([len(forecasts[key].yhat), n_cols - 1])
    199     new_mat[:, 0] = forecasts[key].yhat
    200 

/home/Gopal/anaconda/envs/time_series/lib/python3.6/site-packages/pandas/core/generic.py in __getattr__(self, name)
   5137             if self._info_axis._can_hold_identifiers_and_holds_name(name):
   5138                 return self[name]
-> 5139             return object.__getattribute__(self, name)
   5140 
   5141     def __setattr__(self, name: str, value) -> None:

AttributeError: 'Series' object has no attribute 'yhat'

What could be the best fix for this problem?

https://github.com/carlomazzaferro/scikit-hts/blob/a191eb080d881a071812b539a5bf84f49c5043cd/hts/core/regressor.py#L271

This line results in: ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

I guess it should be elif exogenous_df is not None so that everything works.

Describe the bug

class method _y_hat_matrix seems like filter out leaves part of forecasts, but use keys generated by range as int, so when call y_hat_matrix in functions.py will yield error of can’t find key, because the key of forecasts may be str from df column names.

Additional context to tackle it. should resort the key to make root key in the first (the order is important) and perform simple edit.

Is your feature request related to a problem? Please describe. If I want to create predictions with multiple different revision methods I need to repeatedly start over, define a new regressor, and retrain the all of the models.

Describe the solution like The revision method only comes into play during the prediction stage and does not alter the training step whatsoever. Therefore, I think it should be an option to pass the revision method name to the predict function which will supercede the method specified to the htsregressor object.

Describe alternatives you've considered You can set model.revision_method.name = 'OLS' before calling model.predict( ) to change what revision method is used. I don't think this is the best way to do it.

Additional context Let me know your thoughts. I can implement this ASAP. Should a method be specified when instantiating the regressor object at all?

ValueError ValueError: shapes (533,533) and (505,) not aligned: 533 (dim 1) != 505 (dim 0)

The shape of my modeling data frame is 42 rows × 505 columns.

clf = HTSRegressor(model='prophet', revision_method='OLS', n_jobs=12)
model = clf.fit(model_df, hierarchy)

The fit function is working fine and doing the 100% training. But when I am doing

preds = model.predict(steps_ahead=3)

I am getting value error after the 100% training is done. I have printed each step I am getting output till new_mat and hat_mat

def project(hat_mat: np.ndarray, sum_mat: np.ndarray, optimal_mat: np.ndarray) -> np.ndarray:
    new_mat = np.empty([hat_mat.shape[0], sum_mat.shape[0]])
    print(f"hat_mat: {hat_mat}")
    for i in range(hat_mat.shape[0]):
        new_mat[i, :] = np.dot(optimal_mat, np.transpose(hat_mat[i, :]))
    return new_mat

Describe the bug I am unable to obtain predictions from a FBProphetModel when transform=True. The error message implies that it is unable to safely apply the inv_boxcox method to the values returned by the Prophet model. I suspect this is because the model is returning negative values. To Reproduce

from hts import HTSRegressor
from hts.utilities.load_data import load_mobility_data
df = load_mobility_data()
hier = {
        'total': ['CH', 'SLU', 'BT', 'OTHER'],
        'CH': ['CH-07', 'CH-02', 'CH-08', 'CH-05', 'CH-01'],
        'SLU': ['SLU-15', 'SLU-01', 'SLU-19', 'SLU-07', 'SLU-02'],
        'BT': ['BT-01', 'BT-03'],
        'OTHER': ['WF-01', 'CBD-13']
    }
reg = HTSRegressor(model='prophet', transform=True)
reg = reg.fit(df=df, nodes=hier)
reg.predict(num_steps=30)
Traceback:
...
TypeError: ufunc 'inv_boxcox' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Expected behavior When calling the underlying Prophet model, the input data should be transformed to ensure that the predicted values are positive

Desktop (please complete the following information):

• OS: Linux arch-5.5.7
• scikit-hts version: 0.5.1
• Python version: 3.7.6

Additional context I've tried forcing prophet to fit to positive values using

reg = HTSRegressor(model='prophet', capacity_min=0, capacity_max=10000)
reg.fit(df, nodes=hier)
reg.predict(steps_ahead=30)

which fails because the underlying dataframes have no 'cap' value.

• scikit-hts version: 0.5.5
• Python version: 3.6
• Operating System: Windows

Description

Want to make my predictions to be strictly positive. For that, I want to make use of the log transformation function and I've passed a custom transformation function to the transform parameter. The final results still have negative forecasts. I want to know if I'm passing the custom function correctly. If yes, want to know why the results are negative?.

I've created the custom function in 2 ways. Here goes the first one.

from collections import namedtuple
transform = namedtuple('transform', {'func': 'Callable', 'inv_func': 'Callable'})
transformer = transform(np.log1p, np.exp)
htsmodel = hts.HTSRegressor(model = 'auto_arima', revision_method = 'BU', n_jobs = 0, transform = transformer)

And the second one

transformer = hts._t.Transform(np.log1p, np.exp)
htsmodel = hts.HTSRegressor(model = 'auto_arima', revision_method = 'BU', n_jobs = 0, transform = transformer)

Unfortunately, None of them have any effect in bringing out the positive forecasts. So, Am I passing them the wrong way? or It's something else.

PS: When I'm transforming them explicitly before fitting and back-transforming them after predictions. The forecasts are positive.

Hi, (Seems that I opened the issue with the wrong repo, at scikit-hts-examples, and got no response since Jan 4. But so glad I found this one)

I am following the hts documentation to add exogenous variables to a model.

So far I succeeded in using the hmv load_mobility_data(), creating exogenous features for each node: exogenous = {k: ['precipitation', 'temp'] for k in hmv.columns if k not →in ['precipitation', 'temp']} and pass it in

clf = HTSRegressor(model='prophet', revision_method='OLS',  n_jobs=10)
model = clf.fit(hmv, hier, exogenous=exogenous)

The model ran fine, but I can't figure out how to pass in the exogenous_df in the predict() function. The documentation says:

Parameters
• exogenous_df (pandas.DataFrame) – A dataframe of length == steps_ahead containing the exogenous data for each of the nodes

For the hmv data, the exogenous features were ["temperature", "precipitation"] so I passed in the data frame of 7 rows for predicting the next 7 days as exogenous_df:

ds	     precipitation     temp
2016-09-01	0.00000	    77.00000
2016-09-02	0.00000	    74.00000
2016-09-03	0.00000	    66.00000
2016-09-04	0.00000	    68.00000
2016-09-05	0.00000	    68.00000
2016-09-06	0.00000	    64.00000
2016-09-07	0.00000	    65.00000

and ran preds = model.predict(steps_ahead=7, exogenous_df=exogenous_df)

But I'm not sure how to pass in the node information in the data frame. The hierarchy of the hmv data is:

{'total': ['CH', 'SLU', 'BT', 'OTHER'],
 'CH': ['CH-07', 'CH-02', 'CH-08', 'CH-05', 'CH-01'],
 'SLU': ['SLU-15', 'SLU-01', 'SLU-19', 'SLU-07', 'SLU-02'],
 'BT': ['BT-01', 'BT-03'],
 'OTHER': ['WF-01', 'CBD-13']}

The error I got (which is probably related to the missing node info in the above exogenous_df) is:

~/analytics-etl/virtualenv/lib/python3.8/site-packages/hts/core/regressor.py in predict(self, exogenous_df, steps_ahead, distributor, disable_progressbar, show_warnings, **predict_kwargs)
    270         """
    271 
--> 272         steps_ahead = self.__init_predict_step(exogenous_df, steps_ahead)
    273         predict_function_kwargs = {'fit_kwargs': predict_kwargs,
    274                                    'steps_ahead': steps_ahead,

~/analytics-etl/virtualenv/lib/python3.8/site-packages/hts/core/regressor.py in __init_predict_step(self, exogenous_df, steps_ahead)
    222 
    223     def __init_predict_step(self, exogenous_df: pandas.DataFrame, steps_ahead: int):
--> 224         if self.exogenous and not exogenous_df:
    225             raise MissingRegressorException(f'Exogenous variables were provided at fit step, hence are required at '
    226                                             f'predict step. Please pass the \'exogenous_df\' variable to predict '

~/analytics-etl/virtualenv/lib/python3.8/site-packages/pandas/core/generic.py in __nonzero__(self)
   1327 
   1328     def __nonzero__(self):
-> 1329         raise ValueError(
   1330             f"The truth value of a {type(self).__name__} is ambiguous. "
   1331             "Use a.empty, a.bool(), a.item(), a.any() or a.all()."

ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Can someone provide an example of exogenous_df for the predict() function?

Thank you!

Added the capability to create a grouped time series summing matrix. The methodology takes a slightly different string-based approach to build the summing matrix by string searching the node's keys within the tree, based on the assumption that nodes keys are delimited by underscores(could be any character) and the levels in the hierarchy are unique. I would love your feedback or any further discussion.

If this is a duplicate or already possible in the library, please let me know and I'd be happy to update the documentation to help others like myself with some guidance.

I have included unit tests.

I have not updated the documentation, since it will need to be updated in a couple of places/ examples. Although, I am happy to make the updates based on feedback.

Added Github Action to automate tests on different Python versions/ platforms:

1. Ubuntu Latest - 3.6, 3.7, 3.8
2. Mac OS - 3.6, 3.7

Partially resolves #4.

Here are some additional features not in the PR. I would love to discuss which features should be added prior to PR being merged. Or if they are better addressed in future PRs. They're all not as trivial as adding them to the matrix specification. Caching may be the most copy/paste like. 1 & 3 may require additional investigation in to dependency specification.

1. Expand matrix to Windows
2. Caching of Python dependencies.
3. Python 3.5 compatibility

I am using optimal_combination function with OLS method on the forecast to reconcile it. For some datasets I am getting zero forecasts for all middle levels time series. Why is it so? What can I do to fix it? Can anyone please help.

Using this code https://scikit-hts.readthedocs.io/en/latest/usage.html#reconcile-pre-computed-forecasts

Describe the bug A model trained with week-level data is forecasting steps ahead at day granularity.

To Reproduce I put a reproducible example here - https://stats.stackexchange.com/questions/598669/forecasts-are-at-different-levels-using-htsregressor-hierarchical-time-series

Expected behavior Forecasted output to be same as week-level.

Desktop (please complete the following information):

• OS: [e.g. Monterey]
• scikit-hts version: [e.g. 0.5.12]
• Python version: [e.g. 3.9.12]

Is your feature request related to a problem? Please describe. The predict function of the time series models, auto_arima for example, does not support returning confidence interval for the hierarchical forecast.

Describe the bug I am failing to fit a SarimaxModel because the model reports that the exogenous and endogenous dataframes do not have the same index despite coming from the same dataframe.

After debugging, this seems to be due to the fact that the _get_transformed_data method does not preserve the index using the default transformer

I have been able to correct this using

    def _get_transformed_data(
        self, as_series: bool = False
    ) -> Union[pandas.DataFrame, pandas.Series]:
        key = self.node.key
        value = self.node.item
        transformed = self.transform_function.transform(value[key])
        if as_series:
            return pandas.Series(transformed, index=value[key].index)
        else:
            return pandas.DataFrame({key: transformed})

But perhaps it would be better to modify FunctionTransformer.transform

    def transform(self, x: pandas.Series):
        return self.func(x.values)

as this does not preserve the index of the input series

To Reproduce

from hts.model import SarimaxModel
tree = ...
model = SarimaxModel(node=tree)

#raises ValueError: The indices for endog and exog are not aligned Expected behavior No error should be raised because the two dataframes should have the same index

• scikit-hts version:
• Python version:
• Operating System:

Description

Trying to fit a dataset. dataset is structured properly, as hierarchy tree is properly built

(

ht = hts.hierarchy.HierarchyTree.from_nodes(nodes=hierarchy, df=df)

does not give any error and the tree is built properly how I want.)

No error while defining the model either. (

clf = HTSRegressor(model='prophet', revision_method='BU', n_jobs=2)

gives no error)

But when I try to fit my data, it throws AttributeError saying: 'NoneType' object has no attribute 'fit'

What I Did

clf.fit(df, hierarchy)

• scikit-hts version:0.5.11
• Python version:3.7
• Operating System:Windows

Description

I am trying to run an fbprophet model on hierarchical time series data. It was working fine till now. All of a sudden it is showing pickle file not found for a particular category

What I Did

kwargs = {'yearly_seasonality':True,'weekly_seasonality':True,'changepoint_prior_scale':0.5} clf = HTSRegressor(model='prophet', revision_method='WLSS', n_jobs=4,**kwargs,low_memory=True) model = clf.fit(hierarchy_bottom_level, hierarchy) RemoteTraceback: """ Traceback (most recent call last): File "C:\Users\rahul\anaconda3\envs\py37\lib\multiprocessing\pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "C:\Users\rahul\anaconda3\envs\py37\lib\site-packages\hts\utilities\distribution.py", line 41, in _function_with_partly_reduce return list(results) File "C:\Users\rahul\anaconda3\envs\py37\lib\site-packages\hts\utilities\distribution.py", line 40, in results = (map_function(chunk, kwargs) for chunk in chunk_list) File "C:\Users\rahul\anaconda3\envs\py37\lib\site-packages\hts\core\utils.py", line 57, in _do_actual_fit return _fit_serialize_model(instantiated_model, function_kwargs) File "C:\Users\rahul\anaconda3\envs\py37\lib\site-packages\hts\core\utils.py", line 66, in _fit_serialize_model with open(path, "wb") as p: FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\rahul\AppData\Local\Temp\hts_e76ei9g8\region_B/C & GATING (001).pkl' """