Would it be possible for you to release a new version on pypi with the latest changes from the repository included? Version 0.2.2 is from December 2018 and since then a couple of merge requests have been merged into the master branch which would be nice to have in the pip installable version.

Thanks in advance

Hello. I am receiving this error with

tree interpreter version '0.1.0' and sklearn0.24.0

just after doing from treeinterpreter import treeinterpreter as ti

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-729-679b5145dca2> in <module>
----> 1 from treeinterpreter import treeinterpreter as ti
      2 from sklearn.tree import DecisionTreeRegressor
      3 from sklearn.ensemble import RandomForestRegressor

~/miniconda3/envs/py36_ds_liv/lib/python3.6/site-packages/treeinterpreter/treeinterpreter.py in <module>
      3 import sklearn
      4 
----> 5 from sklearn.ensemble.forest import ForestClassifier, ForestRegressor
      6 from sklearn.tree import DecisionTreeRegressor, DecisionTreeClassifier, _tree
      7 from distutils.version import LooseVersion

ModuleNotFoundError: No module named 'sklearn.ensemble.forest'

How should tests be run?

I get 3 test failures using python setup.py test or pytest using Python 3.6.

======================================================================
FAIL: test_forest_regressor (tests.test_treeinterpreter.TestTreeinterpreter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/c97346/p/treeinterpreter/tests/test_treeinterpreter.py", line 75, in test_forest_regressor
    self.assertTrue(np.allclose(base_prediction, pred))
AssertionError: False is not true

======================================================================
FAIL: test_forest_regressor_joint (tests.test_treeinterpreter.TestTreeinterpreter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/c97346/p/treeinterpreter/tests/test_treeinterpreter.py", line 90, in test_forest_regressor_joint
    self.assertTrue(np.allclose(base_prediction, pred))
AssertionError: False is not true

======================================================================
FAIL: test_tree_regressor (tests.test_treeinterpreter.TestTreeinterpreter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/c97346/p/treeinterpreter/tests/test_treeinterpreter.py", line 40, in test_tree_regressor
    self.assertTrue(np.allclose(base_prediction, pred))
AssertionError: False is not true

I ran the following code, mostly cut-and-pasted from the unit tests, in a python 3 jupyter notebook:

import numpy as np
import unittest

from sklearn.datasets import load_boston, load_iris
from sklearn.ensemble import (RandomForestRegressor, RandomForestClassifier,
                              ExtraTreesClassifier, ExtraTreesRegressor,)
from sklearn.tree import (DecisionTreeClassifier, DecisionTreeRegressor,
                          ExtraTreeClassifier, ExtraTreeRegressor,)

from treeinterpreter import treeinterpreter

boston = load_boston()
iris = load_iris()

TreeRegressor = ExtraTreeRegressor
X = boston.data
Y = boston.target
testX = X[int(len(X)/2):]

#Predict for decision tree
dt = TreeRegressor()
dt.fit(X[:int(len(X)/2)], Y[:int(len(X)/2)])

base_prediction = dt.predict(testX)
pred, bias, contrib = treeinterpreter.predict(dt, testX)

This gave me the following error:

~/.local/lib/python3.5/site-packages/treeinterpreter/treeinterpreter.py in predict(model, X, joint_contribution)
    204     else:
    205         raise ValueError("Wrong model type. Base learner needs to be \
--> 206             DecisionTreeClassifier or DecisionTreeRegressor.")

ValueError: Wrong model type. Base learner needs to be             DecisionTreeClassifier or DecisionTreeRegressor.

Note that running isinstance(dt, DecisionTreeRegressor) returns True. Unfortunately, I don't know enough about python to understand why isinstance would give different answers when it is run in my notebook vs in treeinterpreter.predict.

Do you guys happen to know how to fix this?

A fix for issue #4.

Can be reproduced by running the following:

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from treeinterpreter import treeinterpreter as ti


X = np.array([[0, 0], [1, 1]])
Y = [0, 1]

rf = RandomForestClassifier(n_estimators=10)
rf = rf.fit(X, Y)

prediction, bias, contributions = ti.predict(rf, X)

For some trees in sklearn, the package raises error:

    return _predict_forest(model, X)
  File "python2.7/site-packages/treeinterpreter/treeinterpreter.py", line 96, in _predict_forest
    pred, bias, contribution = _predict_tree(tree, X)
  File "python2.7/site-packages/treeinterpreter/treeinterpreter.py", line 74, in _predict_tree
    biases[row] = values[path[0]]

Error happens when tree value is squeezed into float instead of the 1-D array of floats.

This closes #32 by ensuring that the shape of the output predictions from a DecisionTreeRegressor is a one-dimensional array rather than a 2D (n,1) array.

My bad, I did not properly check #29, and this now causes an

ImportError: cannot import name 'ForestClassifier' from 'sklearn.ensemble' 

upon loading this library.

Sklearn 0.22+ seems to have removed public access to the base classes ForestClassifier and ForestRegressor. This PR changes the type checking to use more concrete implementations RandomForestClassifier, ExtraTreesClassifier, RandomForestRegressor, ExtraTreesRegressor. I think this is a better approach anyway, as sklearn may decide to add more forests type models which we cannot guarantee to support.

From sklearn 0.22 on, importing the treeinterpreter package yields the following FutureWarning:

FutureWarning: The sklearn.ensemble.forest module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.

This PR changes the import to silence this warning, and allow for compatibility with sklearn 0.24 and up.

I tried to use treeinterpreter to calculate feature contribution components for a large dataset which consist of 55K rows, each row ~ 15K features and even though I've parallelized my computation using Spark I was not able to run the code successfully.

One of the issues I was facing was tremendous amount of memory required by treeinterpreter for each run. It turned out that in my case most of the memory is used by _predict_forest to assemble lists of biases, contributions and predictions which are later used to calculate corresponding mean vectors. To improve the code memory usage and runtime I propose to use iterative method for computing averages, as it is summarized in http://www.heikohoffmann.de/htmlthesis/node134.html

Specifically, target support for sklearn's ExtraTreeRegressor, ExtraTreeClassifier, ExtraTreesRegressor, and ExtraTreesClassifier. This is implemented by removing explicit checks on type(model) == DecisionTreeRegressor, etc. and allowing subclasses to fit seamlessly within the treeinterpreter framework by having checks on isinstance(model, DecisionTreeRegressor).

See http://scikit-learn.org/stable/modules/ensemble.html#extremely-randomized-trees for more details on ExtraTrees estimators.

XGBoost and other gradient-boosted estimators do not inherit from the DecisionTreeRegressor or ForestRegressor classes, and thus support is not provided for them through this PR, as seems to have been a concern in previous contributions.

Happens when training on batched data with warm_start = True and the data is unbalanced.

Error:

/Users/x/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py:136: VisibleDeprecationWarning: **Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes)** is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return array(a, dtype, copy=False, order=order, subok=True)
Traceback (most recent call last):
ml-pipeline/src/treeint_simple_example.py", line 22, in <module>
    test_predict_prob, bias, contributions = ti.predict(rf, test_data.head(2))
  File "/Users/x/anaconda3/lib/python3.7/site-packages/treeinterpreter/treeinterpreter.py", line 212, in predict
    return _predict_forest(model, X, joint_contribution=joint_contribution)
  File "/Users/x/anaconda3/lib/python3.7/site-packages/treeinterpreter/treeinterpreter.py", line 166, in _predict_forest
    return (np.mean(predictions, axis=0), np.mean(biases, axis=0),
  File "<__array_function__ internals>", line 6, in mean
  File "/Users/x/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 3373, in mean
    out=out, **kwargs)
  File "/Users/x/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py", line 144, in _mean
    arr = asanyarray(a)
  File "/Users/x/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py", line 136, in asanyarray
    return array(a, dtype, copy=False, order=order, subok=True)
**ValueError: could not broadcast input array from shape (2,1) into shape (2)**

Reproduction:

from sklearn.ensemble import RandomForestClassifier
from treeinterpreter import treeinterpreter as ti
import pandas as pd

# Random forest that can train on chunks of data.
rf = RandomForestClassifier(warm_start=True, n_estimators=1)

# data of chunk1
chunk1_data_vec = [0, 0]
chunk1_df = pd.DataFrame(data={'label': chunk1_data_vec, 'features1': chunk1_data_vec, 'features2': chunk1_data_vec})
# data of chunk2
chunk2_data_vec = [0, 0, 1, 1, 0, 0, 1, 1]
chunk2_df = pd.DataFrame(data={'label': chunk2_data_vec, 'features1': chunk2_data_vec, 'features2': chunk2_data_vec})


# fit first chunk of data that has a single label
rf.fit(X=chunk1_df.drop(['label'], axis='columns'), y=chunk1_df['label'])
# fit second chunk of data that has 2 labels
rf.n_estimators += 1
rf.fit(X=chunk2_df.drop(['label'], axis='columns'), y=chunk2_df['label'])

# test
test_data = chunk2_df.drop(['label'], axis='columns')
# regular predict
rf.predict_proba(test_data)
# tree interpreter predict
test_predict_prob, bias, contributions = ti.predict(rf, test_data.head(2))

Is there a minimum Python version recommended for this package? Would be useful to know it for future conda-forge builds, see for instance https://github.com/conda-forge/treeinterpreter-feedstock/pull/1#issuecomment-757550583

Currently, we couln't able to use the sklearn's pipeline objects directly in treeinterpreter.

possible work around is provided here. Can it be possible to support it natively?

I still get the sklearn deprecation warning about sklearn.ensemble.forest, and when I ran pip install --upgrade treeinterpreter, I got the message that I was already up-to-date.

Hi,

I'm working on a project where treeinterpreter is taking ~2 minutes per prediction.

I suppose this is because the implementation is in pure Python.

Do you know if anyone has looked at porting this to C (or cython or whatever) to make it go faster?

Jim