Hello Tushar,

I am trying to use XBNet for Regression analysis in my research work. Is there any prototype for the application of XBNet when in comes to Regression analysis? I see there is a class XBNETRegressor but would be helpful if there is an example like that for Classifier. Any pointers would also be appreciated.

Thanks & Regards, Rajat

Is there any comparison against other algorithms like XGBoost?

In this paper, XGBoost is compared to other networks:

https://arxiv.org/pdf/2106.03253.pdf

Thank you.

I rebuild the algorithm,but at the "self.boosted_layers[i] = torch.from_numpy(np.array(self.xg.fit(x0.detach().numpy(), (self.l).detach().numpy()).feature_importances_) + self.epsilon)", I can not understand , that feature_importances_ will not change . I think that the function of epsilon does not work.

Hi Tushar. Thanks for sharing the package. I am facing an issue with the line self.temp1 = XGBClassifier().fit(self.X, self.y).feature_importances_ in the method base_tree() of models.py. I can't get much out of the error which says:

UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].
  warnings.warn(label_encoder_deprecation_msg, UserWarning)

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

The surprising thing is that when I run XGBClassifier().fit(X, y) on the same data in an IPython console, it runs fine.

I am using the same script as provided in the README:

import torch
import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from XBNet.training_utils import training,predict
from XBNet.models import XBNETClassifier
from XBNet.run import run_XBNET

# Experiment with Iris data directly from sklearn.
# iris = load_iris()
# data = pd.DataFrame(iris.data)
# data.columns = iris.feature_names
# data.loc[:, 'type'] = iris.target
# X, y = data.iloc[:, :-1], data.iloc[:, -1]
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=123)

data = pd.read_csv('test/Iris.csv')
print(data.shape)
x_data = data[data.columns[:-1]]
print(x_data.shape)
y_data = data[data.columns[-1]]
le = LabelEncoder()
y_data = np.array(le.fit_transform(y_data))
print(le.classes_)

X_train,X_test,y_train,y_test = train_test_split(x_data.to_numpy(),y_data,test_size = 0.3,random_state = 0)
model = XBNETClassifier(X_train,y_train,2)

criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

m,acc, lo, val_ac, val_lo = run_XBNET(X_train,X_test,y_train,y_test,model,criterion,optimizer,32,300)
print(predict(m,x_data.to_numpy()[0,:]))

Any idea what could be going wrong here? Following are some details of my system that might be useful:

• OS: macOS 11.5.2
• Python 3.8.11
• XGBoost version: 1.4.2
• XBNet: 1.3.1

Please update eval_metric from 'merror' to 'mlogloss'

WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'multi:softprob' was changed from 'merror' to 'mlogloss'. Explicitly set eval_metric if you'd like to restore the old behavior.

I tried to run this package on colab facing the below error, when I try to import the run_XBNE

from XBNet.run import run_XBNE

ImportError: cannot import name 'run_XBNE' from 'XBNet.run' (/usr/local/lib/python3.7/dist-packages/XBNet/run.py)

Bumps numpy from 1.21.2 to 1.22.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

While in the code implemented, the minimum value of the gradient is taken into account, and subsequently, the gradient of the layer is updated with the feature importance.

However, in the paper, the algorithm states that the minimum value of the weight of the layer should be taken, and accordingly, the weight matrix of the layer should be updated.

@tusharsarkar3 @dishaShah01 Please clarify this confusion. Also, if possible, please guide through the equation opted for the updation.

Thank you so much.

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting git+https://github.com/tusharsarkar3/XBNet.git Cloning https://github.com/tusharsarkar3/XBNet.git to /tmp/pip-req-build-pmimf9mo Running command git clone -q https://github.com/tusharsarkar3/XBNet.git /tmp/pip-req-build-pmimf9mo Requirement already satisfied: sklearn==0.0 in /usr/local/lib/python3.7/dist-packages (from XBNet==1.4.6) (0.0) ERROR: Could not find a version that satisfies the requirement numpy==1.22.0 (from xbnet) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0rc1, 1.17.0rc2, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0rc1, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0rc1, 1.19.0rc2, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0rc1, 1.20.0rc2, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0rc1, 1.21.0rc2, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6) ERROR: No matching distribution found for numpy==1.22.0

Could you clarify, is there only 1 non-linear layer in your network architecture at the end?

As far as I understood, we are sequentially adding nn.Linear layers to the Sequential list, then, there rises a choice whether to add the activation function (in your Iris example (opened issue), you did not add any activation function, therefore, in fact the whole network is linear in fact). Is it the only non-linearity existing in the network?

Hey I wanted to know if I can use a data generator for data that is too large to fit into memory. I know XGBoost and other similar gradient boosted models have a very slow implementation of to read and re-read a CSV into memory for large datasets but that is much too slow for me. I was hoping I could use a generator with XBNet and read a large H5 file of tab data into memory iteratively and train on the entire dataset the way keras models do it.

Hello,

I am trying to recreate your results for the Iris dataset. I have installed XBNET and all the required libraries.

I am following the example code provided in the READ ME and I am having some trouble with running the code.

Could you please let me know what input and output dimensions you used for layers 1 and 2, with their respective biases, as well as what your last layer was?

Thank you very much.