Added support for custom loss functions in TFFMClassifier. Implemented cross-entropy in the utils module. Also added a class_weight parameter, which automatically uses weighted cross-entropy. One can use class_weight = "balanced" to use the heuristic pos_weight = n_negative / n_positive, where n_positive is the number of positive samples in the training labels.

Note that custom loss functions are disallowed for TFFMRegressor at the moment. Adding them would be trivial, but I don't know of a use case where you'd want something other than MSE, so I left it alone.

I will add a demo of this functionality to the example notebook sometime soon.

Thanks for the great repo, which inspired me on learning tensorflow, I am trying to understand the tffm code line by line.

I am planning to implement the regression part. It's not easy for me, but I would like to give it a try. If anybody have finished this, please give me some hint.

The code for computing predictions (https://github.com/geffy/tffm/blob/master/tffm.py#L212) is incorrect.

You naively applied Lemma 3.1 from Rendle's original paper (http://www.csie.ntu.edu.tw/~b97053/paper/Rendle2010FM.pdf) but this is incorrect when order >= 3.

If you compare with the predictions obtained by Equation (5) in the paper, you'll see that the predictions are not the same.

Hi,

I tried to run TFFMClassifier on a sparse data, (for example: https://github.com/apache/spark/blob/master/data/mllib/sample_libsvm_data.txt), but got an error when fitting the data, the data is loaded with load_svmlight_file ========================= trace log ======================== tensorflow.python.framework.errors.InvalidArgumentError: NaN or Inf in target value : Tensor had NaN values [[Node: target/CheckNumerics = CheckNumericsT=DT_FLOAT, _class=["loc:@add"], message="NaN or Inf in target value", _device="/job:localhost/replica:0/task:0/cpu:0"]] Caused by op u'target/CheckNumerics', defined at: File "/usr/local/lib/python2.7/dist-packages/tffm/testtffm.py", line 51, in model.fit(X_tr.toarray(), y_tr, show_progress=True) File "/usr/local/lib/python2.7/dist-packages/tffm/tffm/base.py", line 242, in fit self.core.build_graph() File "/usr/local/lib/python2.7/dist-packages/tffm/tffm/core.py", line 208, in build_graph self.init_target() File "/usr/local/lib/python2.7/dist-packages/tffm/tffm/core.py", line 191, in init_target msg='NaN or Inf in target value', name='target') File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/numerics.py", line 42, in verify_tensor_all_finite

verify_input = array_ops.check_numerics(t, message=msg)

it seems the problem is in self.loss = self.loss_function(self.outputs, self.train_y), that somehow generated NaN points.

Can someone look at this issue? thanks.

Was poking around and noticed this wasn't installable. Tensorflow isn't on PyPI AFAIK, so this will fail if it's not installed but the error message is ok. Might want to add a note to the README. Not sure what you want to put for version and author information.

Hi geffy! I did a minor change to make it possible to do the prediction in a different batch size than the batch size that is used for training.

This is useful when you want to predict the scores for a given matrix (for example test set matrix) in one go.

Best, Babak.

hello geffy, tffm is very useful, here I have a small question: how do you deal with the l2 regularization when input is sparse, I mean I look at your code, your regularization term include all of the parameters, but when the input data is very sparse, factorization machine only use a small part of parameters(e.g. not all of the w at order 1 can be used), if you do like this, all of the parameters will change, instead of only changing those part of parameters. Do you think it is a issue?

is the self.outputs in the core.py the same as:

from the original FM model paper?

I have trained a model using tffm, and I want to use saved model to inference by c++ code, so I need to rewrite a output by myself, I find the output in your model is a little confusing, so does it have a equation for order 2 FM model like above one? or is it the same equation?

I am trying to understand how tffm is working but I can't figure out why self.loss is obtained by multiplying self.loss_function by self.train_w in the class TFFMCore. I would have thought that self.train_w shouldn't be there...

    def init_loss(self):
        with tf.name_scope('loss') as scope:
            self.loss = self.loss_function(self.outputs, self.train_y) * self.train_w
            self.reduced_loss = tf.reduce_mean(self.loss)
            tf.summary.scalar('loss', self.reduced_loss)

Hi! I just want to ask about do we need to transform every feature column in the dataset to 0/1 representation? I know we need to transform the categorical variables, but what about the numerical variables (like price)? Do we also need to transform them?

Besides, when I tried to transform all variables to 0/1 representations, I got 550+ columns, and I also have 100,000 rows. When I train the model, I always got this error: NaN or Inf in w[2]. : Tensor had NaN values But I am pretty sure there are no other numbers other than 0/1. How does it happen? However, when I only use 90,000 rows of my dataset, this problem disappears.I really don't know why and I really need your help!!!

Thank a lot!!! Weisi

I noticed README mentions TF 1.0 but thought report this and if easy I can fix it. So running test.py for TF 1.3 results in errors below and it seems decision_function() has changed:

======================================================================
ERROR: test_dense_FM (__main__.TestFM)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 54, in test_dense_FM
    self.decision_function_order_4(input_type='dense', use_diag=False)
  File "test.py", line 48, in decision_function_order_4
    actual = model.decision_function(X)
TypeError: decision_function() takes exactly 3 arguments (2 given)

======================================================================
ERROR: test_dense_PN (__main__.TestFM)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 57, in test_dense_PN
    self.decision_function_order_4(input_type='dense', use_diag=True)
  File "test.py", line 48, in decision_function_order_4
    actual = model.decision_function(X)
TypeError: decision_function() takes exactly 3 arguments (2 given)

======================================================================
ERROR: test_sparse_FM (__main__.TestFM)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 60, in test_sparse_FM
    self.decision_function_order_4(input_type='sparse', use_diag=False)
  File "test.py", line 48, in decision_function_order_4
    actual = model.decision_function(X)
TypeError: decision_function() takes exactly 3 arguments (2 given)

======================================================================
ERROR: test_sparse_PN (__main__.TestFM)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 63, in test_sparse_PN
    self.decision_function_order_4(input_type='sparse', use_diag=True)
  File "test.py", line 48, in decision_function_order_4
    actual = model.decision_function(X)
TypeError: decision_function() takes exactly 3 arguments (2 given)

----------------------------------------------------------------------
Ran 4 tests in 8.957s

FAILED (errors=4)

I used sparse style and the following error occurs: `AttributeError Traceback (most recent call last) in ----> 1 model.fit(X_tr, y_tr, show_progress=True)

~\anaconda3\lib\site-packages\tffm\models.py in fit(self, X, y, sample_weight, n_epochs, show_progress) 124 def fit(self, X, y, sample_weight=None, n_epochs=None, show_progress=False): 125 sample_weight = np.ones_like(y) if sample_weight is None else sample_weight --> 126 self.fit(X=X, y_=y, w_=sample_weight, n_epochs=n_epochs, show_progress=show_progress) 127 128 def predict(self, X, pred_batch_size=None):

~\anaconda3\lib\site-packages\tffm\base.py in fit(self, X, y_, w_, n_epochs, show_progress) 224 # iterate over batches 225 for bX, bY, bW in batcher(X_[perm], y_=y_[perm], w_=w_[perm], batch_size=self.batch_size): --> 226 fd = batch_to_feeddict(bX, bY, bW, core=self.core) 227 ops_to_run = [self.core.trainer, self.core.target, self.core.summary_op] 228 result = self.session.run(ops_to_run, feed_dict=fd)

~\anaconda3\lib\site-packages\tffm\base.py in batch_to_feeddict(X, y, w, core) 79 # sparse case 80 X_sparse = X.tocoo() ---> 81 fd[core.raw_indices] = np.hstack( 82 (X_sparse.row[:, np.newaxis], X_sparse.col[:, np.newaxis]) 83 ).astype(np.int64)

AttributeError: 'TFFMCore' object has no attribute 'raw_indices'`

This is not an issue but more of a question. Is it so that tffm supports SGD but not ALS and MCMC algorithms? This was my understanding with a quick look at the code and readme.

pip install tensorflow==2.0 import numpy as np import tensorflow as tf

from tffm import TFFMClassifier

gives error:

AttributeError Traceback (most recent call last) in ----> 1 from tffm import TFFMClassifier

~/anaconda3/lib/python3.7/site-packages/tffm/init.py in ----> 1 from .models import TFFMClassifier, TFFMRegressor 2 3 all = ['TFFMClassifier', 'TFFMRegressor']

~/anaconda3/lib/python3.7/site-packages/tffm/models.py in 2 3 import numpy as np ----> 4 from .base import TFFMBaseModel 5 from .utils import loss_logistic, loss_mse, sigmoid 6

~/anaconda3/lib/python3.7/site-packages/tffm/base.py in 1 import tensorflow as tf ----> 2 from .core import TFFMCore 3 from sklearn.base import BaseEstimator 4 from abc import ABCMeta, abstractmethod 5 import six

~/anaconda3/lib/python3.7/site-packages/tffm/core.py in 4 5 ----> 6 class TFFMCore(): 7 """This class implements underlying routines about creating computational graph. 8

~/anaconda3/lib/python3.7/site-packages/tffm/core.py in TFFMCore() 94 """ 95 def init(self, order=2, rank=2, input_type='dense', loss_function=None, ---> 96 optimizer=tf.train.AdamOptimizer(learning_rate=0.01), reg=0, 97 init_std=0.01, use_diag=False, reweight_reg=False, 98 seed=None):

AttributeError: module 'tensorflow_core._api.v2.train' has no attribute 'AdamOptimizer'

I ran the model on my data and could see huge variations in predictions with respect to previous training on the same data. How can I tackle that so that the prediction is as good as possible?