Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

Overview

Transformers4Rec | Documentation

Build GitHub Documentation GitHub release

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

It works as a bridge between NLP and recommender systems by integrating with one the most popular NLP frameworks HuggingFace Transformers, making state-of-the-art Transformer architectures available for RecSys researchers and industry practitioners.

Sequential and Session-based recommendation with Transformers4Rec
Sequential and Session-based recommendation with Transformers4Rec

Transformers4Rec supports multiple input features and provides configurable building blocks that can be easily combined for custom architectures.

You can build a fully GPU-accelerated pipeline for sequential and session-based recommendation with Transformers4Rec and its smooth integration with other components of NVIDIA Merlin: NVTabular for preprocessing and Triton Inference Server.

Highlights

  • Winning and SOTA solution: We have leveraged and evolved the Transformers4Rec library to win two recent session-based recommendation competitions: the WSDM WebTour Workshop Challenge 2021, organized by Booking.com, and the SIGIR eCommerce Workshop Data Challenge 2021, organized by Coveo. Furthermore, we have also done extensive empirical evaluation on the usage of Transformers4Rec for session-based recommendation, which was able to provide higher accuracy than baselines algorithms, as published in our ACM RecSys'21 paper.

  • Flexibility: The building blocks are modularized and are compatible with vanilla PyTorch modules and TF Keras layers. You can create custom architectures, e.g. with multiple towers, multiple heads/tasks and losses.

  • Production-ready: Exports trained models to serve with Triton Inference Server in a single pipeline that includes online features preprocessing and model inference.

  • Leverages cutting-edge NLP research: With the integration with HuggingFace Transformers, you have available more than 64 different Transformer architectures (and counting) to evaluate for your sequential and session-based recommendation task.

  • Support for multiple input features: HF Transformers supports only sequence of token id as input, as it was originally designed for NLP. Due to the rich features available in RecSys datasets, transformers4Rec enables the usage of HF Transformers with any type of sequential tabular data. The library uses a schema format to configure the input features, and automatically creates the necessary layers (e.g. embedding tables, projection layers, output layers based on the target) without requiring code changes to include new features. Interaction and sequence-level input features can be normalized and combined in configurable ways.

  • Seamless preprocessing and feature engineering: The integration with NVTabular has common preprocessing ops for session-based recommendation and exports a dataset schema compatible with Transformers4Rec, so that input features can be configured automatically.

GPU-accelerated Sequential and Session-based recommendation
GPU-accelerated pipeline for Sequential and Session-based recommendation using NVIDIA Merlin components

Quick tour

To train a model on a dataset, the first step is to provide the schema and use this to construct an input-module. For session-based recommendation problems you typically want to use TabularSequenceFeatures, which merges context features with sequential features. Next, we need to provide the prediction-task(s) (the tasks we provide out of the box can be found here). Then all that's left is to construct a transformer-body and convert this to a model.

Here is the PyTorch version:

from transformers4rec import torch as tr

schema: tr.Schema = tr.data.tabular_sequence_testing_data.schema
# Or read schema from disk: tr.Schema().from_json(SCHEMA_PATH)
max_sequence_length, d_model = 20, 64

# Define input module to process tabular input-features
input_module = tr.TabularSequenceFeatures.from_schema(
    schema,
    max_sequence_length=max_sequence_length,
    continuous_projection=d_model,
    aggregation="concat",
    masking="causal",
)
# Define one or multiple prediction-tasks
prediction_tasks = tr.NextItemPredictionTask()

# Define a transformer-config, like the XLNet architecture
transformer_config = tr.XLNetConfig.build(
    d_model=d_model, n_head=4, n_layer=2, total_seq_length=max_sequence_length
)
model: tr.Model = transformer_config.to_torch_model(input_module, prediction_tasks)

And here is the equivalent code for TensorFlow:

from transformers4rec import tf as tr

schema: tr.Schema = tr.data.tabular_sequence_testing_data.schema
# Or read schema from disk: tr.Schema().from_json(SCHEMA_PATH)
max_sequence_length, d_model = 20, 64

# Define input module to process tabular input-features
input_module = tr.TabularSequenceFeatures.from_schema(
    schema,
    max_sequence_length=max_sequence_length,
    continuous_projection=d_model,
    aggregation="concat",
    masking="causal",
)
# Define one or multiple prediction-tasks
prediction_tasks = tr.NextItemPredictionTask()

# Define a transformer-config, like the XLNet architecture
transformer_config = tr.XLNetConfig.build(
    d_model=d_model, n_head=4, n_layer=2, total_seq_length=max_sequence_length
)
model: tr.Model = transformer_config.to_tf_model(input_module, prediction_tasks)

Use cases

Sequential and Session-based recommendation

Traditional recommendation algorithms usually ignore the temporal dynamics and the sequence of interactions when trying to model user behaviour. Generally, the next user interaction is related to the sequence of the user's previous choices. In some cases, it might be even a repeated purchase or song play. User interests might also suffer from the interest drift, as preferences might change over time. Those challenges are addressed by the sequential recommendation task. A special case of sequential-recommendation is the session-based recommendation task, where you have only access to the short sequence of interactions within the current session. This is very common in online services like e-commerce, news and media portals where the user might choose to browse anonymously (and due to GDPR compliance no cookies are collected), or because it is a new user. This task is also relevant for scenarios where users' interests change a lot over time depending on the user context or intent, so leveraging the current session interactions is more promising than old interactions to provide relevant recommendations.

To deal with sequential and session-based recommendation, many sequence learning algorithms previously applied in machine learning and NLP research have been explored for RecSys, based on k-Nearest Neighbors, Frequent Pattern Mining, Hidden Markov Models, Recurrent Neural Networks, and more recently neural architectures using the Self-Attention Mechanism and the Transformer architectures.

Differently from Transformers4Rec, existing frameworks for such tasks are generally focused for research, accept only sequence of item ids as input and do not provide a modularized and scalable implementation for production usage.

Installation

Installing with pip

Transformers4Rec comes in two flavors: PyTorch and Tensorflow. It can optionally use the GPU-accelerated NVTabular dataloader, which is highly recommended. Those components can be installed as optional args for the pip install package.

  • All
    pip install transformers4rec[all]
  • PyTorch
    pip install transformers4rec[torch,nvtabular]
  • Tensorflow:
    pip install transformers4rec[tensorflow,nvtabular]

Installing with conda

conda install -c nvidia transformers4rec

Installing with Docker

Transformers4Rec library is pre-installed in the NVIDIA Merlin Docker containers, that are available in the NVIDIA container repository in three different containers:

Container Name Container Location Functionality
merlin-tensorflow-training https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-tensorflow-training Transformers4Rec, NVTabular, TensorFlow, and HugeCTR Tensorflow Embedding plugin
merlin-pytorch-training https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-pytorch-training Transformers4Rec, NVTabular and PyTorch
merlin-inference https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-inference Transformers4Rec, NVTabular, PyTorch, and Triton Inference

To use these Docker containers, you'll first need to install the NVIDIA Container Toolkit to provide GPU support for Docker. You can use the NGC links referenced in the table above to obtain more information about how to launch and run these containers.

Feedback and Support

If you'd like to contribute to the library directly, see the CONTRIBUTING.md. We're particularly interested in contributions or feature requests for our feature engineering and preprocessing operations. To further advance our Merlin Roadmap, we encourage you to share all the details regarding your recommender system pipeline in this survey.

If you're interested in learning more about how NVTabular works, see Transformers4Rec documentation. We also have the API documentation that outlines specifics of the available modules and classes within the Transformers4Rec library.

Issues
  • Rename ColumnGroup to Schema

    Rename ColumnGroup to Schema

    The ColumnGroup abstraction in NVTabular is being phased out. This change:

    • Renames ColumnGroup to Schema
    • Renames Column to ColumnSchema
    • removes the 'tags_to_filter' functionality from the old 'from_column_group' methods (wasn't used and didn't work)
    opened by benfred 6
  • PyTorch installation errors [BUG]

    PyTorch installation errors [BUG]

    Bug description

    I did an install of the PyTorch version of Transformers4Rec and tried to run the example code in the ReadMe. However, the needed torchmetrics library was not automatically installed and needed to be installed by hand. After installing torchmetrics manually following error appeared:

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-4-77b1c0b67bb1> in <module>()
    ----> 1 from transformers4rec import torch as tr
          2 
          3 schema: tr.Schema = tr.data.tabular_sequence_testing_data.schema
          4 # Or read schema from disk: tr.Schema().from_json(SCHEMA_PATH)
          5 max_sequence_length, d_model = 20, 64
    
    8 frames
    /usr/lib/python3.7/importlib/util.py in find_spec(name, package)
        112         else:
        113             if spec is None:
    --> 114                 raise ValueError('{}.__spec__ is None'.format(name))
        115             return spec
        116 
    
    ValueError: transformers.__spec__ is None
    

    Steps/Code to reproduce bug

    1. Install PyTorch version.
    2. Try the example code.

    Expected behavior

    Automatic installation of depenencies.

    Environment details

    • Transformers4Rec version: 0.1.1
    • Platform: Google Colab
    • Python version: 3.7.12
    • Huggingface Transformers version: 4.9.2
    • PyTorch version (GPU?): 1.9.0+cu102
    • Tensorflow version (GPU?): 2.6.0
    bug status/needs-triage 
    opened by pintonos 6
  • [QST] Non-`nvtabular` examples?

    [QST] Non-`nvtabular` examples?

    ❓ Questions & Help

    Details

    Hi all! I'm excited to learn more about transformers4rec. Unfortunately, I'm having trouble following some of the examples due to the library's (likely preferred) way of handling datasets: via nvtabular. As I understand it, nvtabular has a hard dependency on cupy, which itself cannot be installed on machines without a GPU. This makes it difficult to even add as a dependency (at least, via pip/poetry) since at installation time, cupy checks for NVIDIA library headers.

    If that's correct, I'd love to know (or...potentially contribute?) examples that don't use the nvtabular extra. Please let me know if I'm missing something very obvious! Thanks all 🙏🏻

    status/needs-triage 
    opened by dataframing 5
  • Add tutorial folder and preproc nb

    Add tutorial folder and preproc nb

    We decided to host tutorial content in the Transformers4Rec GH repo as well. I am adding the very first nb that generates the Oct-2019.parquet file that is used in the actual tutorial ETL nb.

    P0 area/tutorial 
    opened by rnyak 5
  • Initial version of documentation

    Initial version of documentation

    Documentation of Transformers4Rec: how to install with docker and conda, example commands to run preproc and training, API documentation

    documentation P0 status/won't-fix 
    opened by gabrielspmoreira 4
  • Making PyTorch, Tensorflow & NVTabular optional dependencies

    Making PyTorch, Tensorflow & NVTabular optional dependencies

    null

    API / packaging area/tests status/needs-review 
    opened by marcromeyn 4
  • [BUG] No module named `tensorflow_metadata` during `Schema().from_proto_text()`.

    [BUG] No module named `tensorflow_metadata` during `Schema().from_proto_text()`.

    Bug description

    Steps/Code to reproduce bug

    1. Install transformers4rec with just the pytorch (torch?) extra.

    2. Run the following:

      from merlin_standard_lib import Schema
      
      Schema().from_proto_text("")
      

      This should give you an error like so:

      ---------------------------------------------------------------------------
      
      ModuleNotFoundError                       Traceback (most recent call last)
      
      /tmp/ipykernel_3315/4183349990.py in <module>
            1 from merlin_standard_lib import Schema
            2 
      ----> 3 Schema().from_proto_text("")
            4 
      
      ~/.cache/pypoetry/virtualenvs/tfr-sample-project/lib/python3.7/site-packages/merlin_standard_lib/schema/schema.py in from_proto_text(self, path_or_proto_text)
          400 
          401     def from_proto_text(self, path_or_proto_text: str) -> "Schema":
      --> 402         from tensorflow_metadata.proto.v0 import schema_pb2
          403 
          404         return proto_utils.proto_text_to_better_proto(self, path_or_proto_text, schema_pb2.Schema())
      
      ModuleNotFoundError: No module named 'tensorflow_metadata'
      

    The empty string "" may be an invalid protobuf schema, but it seems like tensorflow_metadata is imported immediately, so I don't think that's the problem.

    Expected behavior

    I was just hoping to probe the Schema building (before I saw that you can do it manually on #271) to better understand what's going on, but hit this issue almost immediately.

    Environment details

    • Transformers4Rec version: 0.1.1 (with pytorch extra)
    • Platform: Ubuntu 18.04
    • Python version: 3.7.10
    • Huggingface Transformers version: 4.9.2
    • PyTorch version (GPU?): 1.9.1+cu102
    • Tensorflow version (GPU?): N/A

    Additional context

    bug status/needs-triage 
    opened by dataframing 4
  • [REVIEW] Ecom-rees preproc with  NVTabular

    [REVIEW] Ecom-rees preproc with NVTabular

    @gabrielspmoreira can you please check the notebook? I'd appreciate if you can run it end to end and see the saved files. Thanks.

    opened by rnyak 4
  • [Docs] Main page and Core features

    [Docs] Main page and Core features

    Closes #164, #167 This PR comprises the main page of the docs (will also be the README.md of the repo) and the core features page. It describes the Transformers4Rec library, its integration with NVTabular and Triton. Also describes sequential and session-based recommendation, the relationship between NLP and RecSys and the integration with HF Transforemrs. Describes also the core features of the library in terms of model definition and training.

    area/documentation status/needs-review 
    opened by gabrielspmoreira 4
  • [BUG] Docker Image Contains NVTabular 0.6.0

    [BUG] Docker Image Contains NVTabular 0.6.0

    Bug description

    Docker image for PyTorch Training contains NVTabular 0.6.0. I'm unable to use the nvt.ops.Categorify(start_index=1). I get an error saying that start_index is not an expected param.

    Steps/Code to reproduce bug

    1. download the docker image
    2. Import some data, and try to categorify nvt.ops.Categorify(start_index=1)

    Expected behavior

    Environment details

    Using Docker container here: https://ngc.nvidia.com/catalog/containers/nvidia:merlin:merlin-pytorch-training Ran command docker run --gpus all --rm -it -p 8888:8888 -p 8797:8787 -p 8796:8786 --ipc=host nvcr.io/nvidia/merlin/merlin-pytorch-training:0.6 /bin/bash

    Additional context

    bug status/needs-triage 
    opened by zanussbaum 4
  • Check results on tensorflow

    Check results on tensorflow

    null

    opened by benfred 0
  • Check results on pytorch

    Check results on pytorch

    null

    opened by benfred 0
  • [WIP] Define getting-started example notebook using TensorFlow & Add support of graph-mode

    [WIP] Define getting-started example notebook using TensorFlow & Add support of graph-mode

    Fixes #283 #262 #258 #236

    Goals :soccer:

    • Create a TensorFlow version of the getting started notebook, including model definition, training, evaluation, saving, and reloading.

    • Add support of graph and eager modes of the following TensorFlow classes: MaskSequence, NextItemPredictionTask, RankingMetric

    • Update HuggingFace dependency to the latest release to include support of Albert and Electra. (Note that ELECTRA model still need feature #263 for working correctly)

    Implementation Details :construction:

    • Upgrade HF library to include support of failing architectures: Albert and ELECTRA

    • Refactor the ops used in MaskSequence, NextItemPredictionTask, RankingMetric classes to ensure graph mode is working. Note that a tf.Variable as a placeholder was created to replace the definition of fixed tensors in these classes.

    • Refactor the input argument of the TranformerBlock call method as all HF transformers support feeding the inputs as a dictionary. Note that the previous call method provided transformer_kwargs twice, which raised an error when saving the HF transformer model (mask_perm was set equal to input_embed instead of None).

    • Expose compute_metrics flag of compute_loss methods to compute metrics during train / evaluation. The method process_metric is added to flatten ranking metrics with respect to their threshold.

    • Fix performance of NextItemPredictionTask: - Set from_logits=True of SparseCategoricalCrossentropy loss as NextItemPredictionTask is returning the logits.

    Testing Details :mag:

    • Add tests to test_model.py: - Add test of sequence classification model and check if loss decreasing - Test, next item prediction task, using eager and graph modes - Test saving of next item prediction task with a simple model using MLP block - Test saving of next item prediction task with supported HF transformers

    • Update test_transformer.py to include Albert model.

    • Add a test to test_ranking_metrics.py to check that computed metrics are different from 0.

    The Two remaining tasks are:

    • Fix scores of ranking metrics returned by the fit/evaluate methods in the getting-started notebook.
    • Fix re-loading model from saved checkpoints.
    opened by sararb 1
  • [Task] Create unit tests for data_loader_engine='pyarrow',  and no_cuda =False

    [Task] Create unit tests for data_loader_engine='pyarrow', and no_cuda =False

    Description

    Currently all our unit tests for data_loader_engine='pyarrow' checks the training and eval_loop on CPU by setting no_cuda=True. We need to check the same on GPU with no_cuda=False.

    Tests / CI / CD area/pytorch 
    opened by rnyak 0
  • PyArrow GPU Evaluation [BUG]

    PyArrow GPU Evaluation [BUG]

    Bug description

    Performing a fit_and_evaluate function call with pyarrow as a dataloader leads to following error:

    ...
    File "/home/.../miniconda3/envs/trans4rec/lib/python3.7/site-packages/torch/nn/functional.py", line 1847, in linear
        return torch._C._nn.linear(input, weight, bias)
    RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking arugment for argument mat2 in method wrapper_mm)
    

    Steps/Code to reproduce bug

    train_args = tr.trainer.T4RecTrainingArguments(
                                                data_loader_engine='pyarrow',
                                                dataloader_drop_last = True,
                                                report_to = [], 
                                                gradient_accumulation_steps = 1,
                                                per_device_train_batch_size = 64, 
                                                per_device_eval_batch_size = 64,
                                                output_dir = "./tmp", 
                                                learning_rate=0.0005,
                                                lr_scheduler_type='cosine', 
                                                learning_rate_num_cosine_cycles_by_epoch=1.5,
                                                num_train_epochs=1,
                                                max_sequence_length=20, 
                                                no_cuda=False)
    
    
    
    trainer = tr.Trainer(
        model=model,
        args=train_args,
        schema=schema,
        compute_metrics=True)
    
    results = fit_and_evaluate(trainer, start_time_index=1, end_time_index=1, input_dir='./data/')
    

    It is worth noting that the training seems to work fine, but the error gets thrown while evaluating the model!

    Environment details

    • Transformers4Rec version: 0.1.1
    • Python version: 3.7
    • Huggingface Transformers version: 4.9.2
    • PyTorch version (GPU?): 1.9.1 GPU
    bug status/needs-triage 
    opened by pintonos 2
  • [BUG] Add tf graph-mode support

    [BUG] Add tf graph-mode support

    Bug description

    Steps/Code to reproduce bug

    1. Define the model for NextItemPrediction
    2. Compile model with graph model: `model.compile(optimizer="adam", run_eagerly=False) More details can be found in this unit test

    Expected behavior

    Making sure the tf pipeline (training, evaluating, and saving the model) is working in graph mode

    bug 
    opened by sararb 0
  • [QST] export model to Torchscript

    [QST] export model to Torchscript

    ❓ Questions & Help

    Details

    I am trying to export the trained model to Torchscript with the following script bu I got an error:

    import torch
    traced_model = torch.jit.trace(model, batch_input)
    
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    /tmp/ipykernel_2414/1419015779.py in <module>
          1 # Creating the trace
          2 import torch
    ----> 3 traced_model = torch.jit.trace(model, batch_input)
    
    /opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
        733 
        734     if isinstance(func, torch.nn.Module):
    --> 735         return trace_module(
        736             func,
        737             {"forward": example_inputs},
    
    /opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
        950             example_inputs = make_tuple(example_inputs)
        951 
    --> 952             module._c._create_method_from_trace(
        953                 method_name,
        954                 func,
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
       1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
       1050                 or _global_forward_hooks or _global_forward_pre_hooks):
    -> 1051             return forward_call(*input, **kwargs)
       1052         # Do not call functions when jit is used
       1053         full_backward_hooks, non_full_backward_hooks = [], []
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
       1037                 recording_scopes = False
       1038         try:
    -> 1039             result = self.forward(*input, **kwargs)
       1040         finally:
       1041             if recording_scopes:
    
    /workspace/TF4Rec/transformers4rec/torch/model/base.py in forward(self, inputs, training, **kwargs)
        520         outputs = {}
        521         for head in self.heads:
    --> 522             outputs.update(head(inputs, call_body=True, training=training, always_output_dict=True))
        523 
        524         if len(outputs) == 1:
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
       1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
       1050                 or _global_forward_hooks or _global_forward_pre_hooks):
    -> 1051             return forward_call(*input, **kwargs)
       1052         # Do not call functions when jit is used
       1053         full_backward_hooks, non_full_backward_hooks = [], []
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
       1037                 recording_scopes = False
       1038         try:
    -> 1039             result = self.forward(*input, **kwargs)
       1040         finally:
       1041             if recording_scopes:
    
    /workspace/TF4Rec/transformers4rec/torch/model/base.py in forward(self, body_outputs, training, call_body, always_output_dict, **kwargs)
        386 
        387         if call_body:
    --> 388             body_outputs = self.body(body_outputs, training=training)
        389 
        390         for name, task in self.prediction_task_dict.items():
    
    /workspace/TF4Rec/transformers4rec/config/schema.py in __call__(self, *args, **kwargs)
         48         self.check_schema()
         49 
    ---> 50         return super().__call__(*args, **kwargs)
         51 
         52     def _maybe_set_schema(self, input, schema):
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
       1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
       1050                 or _global_forward_hooks or _global_forward_pre_hooks):
    -> 1051             return forward_call(*input, **kwargs)
       1052         # Do not call functions when jit is used
       1053         full_backward_hooks, non_full_backward_hooks = [], []
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
       1037                 recording_scopes = False
       1038         try:
    -> 1039             result = self.forward(*input, **kwargs)
       1040         finally:
       1041             if recording_scopes:
    
    /workspace/TF4Rec/transformers4rec/torch/block/base.py in forward(self, input, training, **kwargs)
        149 
        150             elif "training" in inspect.signature(module.forward).parameters:
    --> 151                 input = module(input, training=training)
        152             else:
        153                 input = module(input)
    
    /workspace/TF4Rec/transformers4rec/config/schema.py in __call__(self, *args, **kwargs)
         48         self.check_schema()
         49 
    ---> 50         return super().__call__(*args, **kwargs)
         51 
         52     def _maybe_set_schema(self, input, schema):
    
    /workspace/TF4Rec/transformers4rec/torch/tabular/base.py in __call__(self, inputs, pre, post, merge_with, aggregation, *args, **kwargs)
        377 
        378         # This will call the `forward` method implemented by the super class.
    --> 379         outputs = super().__call__(inputs, *args, **kwargs)  # noqa
        380 
        381         if isinstance(outputs, dict):
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
       1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
       1050                 or _global_forward_hooks or _global_forward_pre_hooks):
    -> 1051             return forward_call(*input, **kwargs)
       1052         # Do not call functions when jit is used
       1053         full_backward_hooks, non_full_backward_hooks = [], []
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
       1037                 recording_scopes = False
       1038         try:
    -> 1039             result = self.forward(*input, **kwargs)
       1040         finally:
       1041             if recording_scopes:
    
    /workspace/TF4Rec/transformers4rec/torch/features/sequence.py in forward(self, inputs, training, **kwargs)
        250 
        251         if self.masking or self.projection_module:
    --> 252             outputs = self.aggregation(outputs)
        253 
        254         if self.projection_module:
    
    /workspace/TF4Rec/transformers4rec/config/schema.py in __call__(self, *args, **kwargs)
         48         self.check_schema()
         49 
    ---> 50         return super().__call__(*args, **kwargs)
         51 
         52     def _maybe_set_schema(self, input, schema):
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
       1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
       1050                 or _global_forward_hooks or _global_forward_pre_hooks):
    -> 1051             return forward_call(*input, **kwargs)
       1052         # Do not call functions when jit is used
       1053         full_backward_hooks, non_full_backward_hooks = [], []
    
    /opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
       1037                 recording_scopes = False
       1038         try:
    -> 1039             result = self.forward(*input, **kwargs)
       1040         finally:
       1041             if recording_scopes:
    
    /workspace/TF4Rec/transformers4rec/torch/tabular/aggregation.py in forward(self, inputs)
         37         inputs: TabularData,
         38     ) -> torch.Tensor:
    ---> 39         self._expand_non_sequential_features(inputs)
         40         self._check_concat_shapes(inputs)
         41 
    
    /workspace/TF4Rec/transformers4rec/torch/tabular/base.py in _expand_non_sequential_features(self, inputs)
         51     def _expand_non_sequential_features(self, inputs: TabularData) -> TabularData:
         52         inputs_sizes = {k: v.shape for k, v in inputs.items()}
    ---> 53         seq_features_shapes, sequence_length = self._get_seq_features_shapes(inputs_sizes)
         54 
         55         if len(seq_features_shapes) > 0:
    
    /workspace/TF4Rec/transformers4rec/torch/tabular/base.py in _get_seq_features_shapes(self, inputs_sizes)
         71         if len(seq_features_shapes) > 0:
         72             if len(set(seq_features_shapes.values())) > 1:
    ---> 73                 raise ValueError(
         74                     "All sequential features must share the same shape in the first two dims "
         75                     "(batch_size, seq_length): {}".format(seq_features_shapes)
    
    ValueError: All sequential features must share the same shape in the first two dims (batch_size, seq_length): {'item_id-list_seq': (tensor(384), tensor(20)), 'category-list_seq': (tensor(384), tensor(20))}
    

    Motivation

    I need to export model as torchscript to be able to test if I can serve it to Triton with Torch backend.

    area/inference status/needs-triage 
    opened by rnyak 2
  • [FEA] Port TabularDropout transformation from PyTorch to TF

    [FEA] Port TabularDropout transformation from PyTorch to TF

    opened by gabrielspmoreira 0
  • [Task] Add Unit test to test TF model with KerasSequenceLoader

    [Task] Add Unit test to test TF model with KerasSequenceLoader

    Description

    Currently we do not have unit tests with TF models using KerasSequenceLoader. We should add it and test if GPU is utilized as expected.

    Tests / CI / CD area/tensorflow area/tests 
    opened by rnyak 0
  • [FEA] How can it be extended to handle next basket recommendation ?

    [FEA] How can it be extended to handle next basket recommendation ?

    ❓ Questions & Help

    Details

    As titled, how can Transformers4Rec be extended to handle next basket recommendation, not just item recommendation?

    status/needs-triage 
    opened by tim5go 5
Releases(v0.1.1)
Owner
Merlin is a framework providing end-to-end GPU-accelerated recommender systems, from feature engineering to deep learning training and deploying to production
null
Mirco Ravanelli 2.1k Oct 18, 2021
Beautiful visualizations of how language differs among document types.

Scattertext 0.1.0.0 A tool for finding distinguishing terms in corpora and displaying them in an interactive HTML scatter plot. Points corresponding t

Jason S. Kessler 1.7k Oct 23, 2021
Beautiful visualizations of how language differs among document types.

Scattertext 0.1.0.0 A tool for finding distinguishing terms in corpora and displaying them in an interactive HTML scatter plot. Points corresponding t

Jason S. Kessler 1.5k Feb 17, 2021
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 52.8k Oct 21, 2021
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 40.9k Feb 18, 2021
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 52.8k Oct 22, 2021
Subtitle Workshop (subshop): tools to download and synchronize subtitles

SUBSHOP Tools to download, remove ads, and synchronize subtitles. SUBSHOP Purpose Limitations Required Web Credentials Installation, Configuration, an

Joe D 1 Oct 22, 2021
Task-based datasets, preprocessing, and evaluation for sequence models.

SeqIO: Task-based datasets, preprocessing, and evaluation for sequence models. SeqIO is a library for processing sequential data to be fed into downst

Google 78 Oct 18, 2021
Graph4nlp is the library for the easy use of Graph Neural Networks for NLP

Graph4NLP Graph4NLP is an easy-to-use library for R&D at the intersection of Deep Learning on Graphs and Natural Language Processing (i.e., DLG4NLP).

Graph4AI 1.1k Oct 22, 2021
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language mod

null 13.2k Jul 7, 2021
Making text a first-class citizen in TensorFlow.

TensorFlow Text - Text processing in Tensorflow IMPORTANT: When installing TF Text with pip install, please note the version of TensorFlow you are run

null 834 Oct 22, 2021
Making text a first-class citizen in TensorFlow.

TensorFlow Text - Text processing in Tensorflow IMPORTANT: When installing TF Text with pip install, please note the version of TensorFlow you are run

null 692 Feb 16, 2021
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language mod

null 14.2k Oct 23, 2021
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language mod

null 11.3k Feb 18, 2021
ChessCoach is a neural network-based chess engine capable of natural-language commentary.

ChessCoach is a neural network-based chess engine capable of natural-language commentary.

Chris Butner 327 Oct 18, 2021
🤗 The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

?? The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Hugging Face 10.2k Oct 22, 2021
PyTorch original implementation of Cross-lingual Language Model Pretraining.

XLM NEW: Added XLM-R model. PyTorch original implementation of Cross-lingual Language Model Pretraining. Includes: Monolingual language model pretrain

Facebook Research 2.5k Oct 15, 2021
Simple and efficient RevNet-Library with DeepSpeed support

RevLib Simple and efficient RevNet-Library with DeepSpeed support Features Half the constant memory usage and faster than RevNet libraries Less memory

Lucas Nestler 74 Oct 15, 2021
Coreference resolution for English, German and Polish, optimised for limited training data and easily extensible for further languages

Coreferee Author: Richard Paul Hudson, msg systems ag 1. Introduction 1.1 The basic idea 1.2 Getting started 1.2.1 English 1.2.2 German 1.2.3 Polish 1

msg systems ag 103 Oct 14, 2021