This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf

Related tags

Deep Learning behavior transformers pytorch seq2seq alibaba recommenders seq2seq-attn behavior-sequence-transformer

Overview

Behavior-Sequence-Transformer-Pytorch

This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf

This model is a novel recommender architecture based on seq2seq models. We translate user behaviour into sequences and predict a rating for each target item (movie).

Dataset

For this implementation we used Movielens 1M Dataset that contains timestamps per each rating, making it perfect to test in the sequence recommendation model.

Running

You can run it in colab here. If you prefer to run locally the model architecture is contained on pytorch-best.ipynb while data processing is on the prepare_data.ipynb notebook and should be run first.

Results

Training on all-1 user ratings and leaving the latest rating for test we obtain the following results

Dataset	MAE	RMSE
Train	0.72	0.84
Test	0.74	0.93

Here is a screenshot of training logs we we see overfitting from epoch 12-15.

References

Original paper 1
Keras implementation 2
Tensorflow implementation 3

Comments

a question about PositionalEmbedding module

Helo，thanks for your complete code. I read this code carefully，and have a question.

It's BST model's forward function

transfomer_features, user_features, target_movie_rating = self.encode_input(batch)
transfomer_features = self.positional_embedding(transfomer_features)

And it's positional_embedding module

class PositionalEmbedding(nn.Module):
    def __init__(self, max_len, d_model):
        super().__init__()

        self.pe = nn.Embedding(max_len, d_model)

    def forward(self, x):
        batch_size = x.size(0)
        return self.pe.weight.unsqueeze(0).repeat(batch_size, 1, 1)

we can see that PositionalEmbedding do nothing, just return it's weight. PositionalEmbedding doesn't have any interaction with x. It's really confused.

opened by watchwater 3

[sequence question] How should I run this code if I set the sequence_length to 4？

Thanks for sharing this code, and I have a question with it.

When I run this code on Colab with the original setup，it would run perfectly. But when I change the sequence_length = 4 and step_size = 2 (just like the description in With sequence_length of 4 and step_size of 2, we end up with 498,623 sequences.), the code has an error with Sizes of tensors must match except in dimension 2. Expected size 4 but got size 8 for tensor number 1 in the list. in line transfomer_features = torch.cat((transfomer_features, positional_embedding), dim=2) What should I do to get it to work properly?

P.S. Sorry for my poor English.

opened by bpbdq 0

Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

CIPS -- Official Pytorch Implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis Requirements pip install -r requi

Multimodal Lab @ Samsung AI Center Moscow

201 Dec 21, 2022

Official Implementation for "ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement" https://arxiv.org/abs/2104.02699

ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement Recently, the power of unconditional image synthesis has significantly advanced th

967 Jan 4, 2023

This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

367 Dec 27, 2022

Minimal implementation of PAWS (https://arxiv.org/abs/2104.13963) in TensorFlow.

PAWS-TF 🐾 Implementation of Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples (PAWS)

43 Jan 8, 2023

Unofficial implementation of "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" (https://arxiv.org/abs/2103.14030)

Swin-Transformer-Tensorflow A direct translation of the official PyTorch implementation of "Swin Transformer: Hierarchical Vision Transformer using Sh

52 Dec 29, 2022

Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Fastformer-Keras Unofficial Tensorflow-Keras implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Tensorflo

10 Jan 30, 2022

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Transfer Learning for Text Classification with Tensorflow Tensorflow implementation of Semi-supervised Sequence Learning(https://arxiv.org/abs/1511.01

82 Oct 22, 2022

This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).

Predicting Patient Outcomes with Graph Representation Learning This repository contains the code used for Predicting Patient Outcomes with Graph Repre

76 Dec 22, 2022

https://arxiv.org/abs/2102.11005

LogME LogME: Practical Assessment of Pre-trained Models for Transfer Learning How to use Just feed the features f and labels y to the function, and yo

149 Dec 19, 2022

This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf

Related tags

Overview

Behavior-Sequence-Transformer-Pytorch

Dataset

Running

Results

References

You might also like...

Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

Official Implementation for "ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement" https://arxiv.org/abs/2104.02699

This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Minimal implementation of PAWS (https://arxiv.org/abs/2104.13963) in TensorFlow.

Unofficial implementation of "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" (https://arxiv.org/abs/2103.14030)

Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

This repository contains the code used for Predicting Patient Outcomes with Graph Representation Learning (https://arxiv.org/abs/2101.03940).

https://arxiv.org/abs/2102.11005

Comments

a question about PositionalEmbedding module

[sequence question] How should I run this code if I set the sequence_length to 4？

Owner

Jaime Ferrando Huertas

Source Code for DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances (https://arxiv.org/pdf/2012.01775.pdf)

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).

We have implemented shaDow-GNN as a general and powerful pipeline for graph representation learning. For more details, please find our paper titled Deep Graph Neural Networks with Shallow Subgraph Samplers, available on arXiv (https//arxiv.org/abs/2012.01380).

Pytorch implementation of Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization https://arxiv.org/abs/2008.11646

[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

A PyTorch implementation of EventProp [https://arxiv.org/abs/2009.08378], a method to train Spiking Neural Networks

Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) in PyTorch

Pytorch implementation of Distributed Proximal Policy Optimization: https://arxiv.org/abs/1707.02286

PyTorch implementation of Asymmetric Siamese (https://arxiv.org/abs/2204.00613)