Job-Recommend-Competition - Vectorwise Interpretable Attentions for Multimodal Tabular Data

Related tags

Deep Learning nlp competition natural-language-processing deep-learning tabular-data pytorch attention-mechanism attention-model multimodal-learning multimodal-deep-learning dacon pytorch-lightning wandb tabnet
Overview

SiD - Simple Deep Model

Hits

Vectorwise Interpretable Attentions for Multimodal Tabular Data

Winning Solution for a Competition

This repository is an official implementation of a model which won first place in the dacon competition. You can see the final result on this post. If you want to reproduce the score of the competition, please check out this documentation which is used to verify by the competition hosts.

Introduction

SiD is a vectorwise interpretable attention model for multimodal tabular data. It is designed and considered to handle data as vectors so that multimodal data (e.g. text, image and audio) can be encoded into the vectors and used with the the tabular data.

Requirements

The requirements of this project is as follows:

  • numpy
  • omegaconf
  • pandas
  • pytorch_lightning
  • scikit_learn
  • torch==1.10.1
  • transformers
  • wandb

Instead, you can simply install the libraries at once:

$ pip install -r requirements.txt

Architecture

Model Architecture Residual Block

As mentioned above, SiD is considered to extend TabNet with vectorwise approach. Because many multimodal data (e.g. text, image and audio) are encoded into the vectors, it is important to merge the tabular data with the vectors. However, the attention mechanism (attentive transformer) of TabNet does not consider the vectorized features. Therefore we propose the vectorwise interpretable attention model.

Experiments

Hyperparameter Settings
Experimental Results
Ablation Studies

Interpretability

2017
2018
2019
2020
Importance Mask
Question Dialogs

License

This repository is released under the Apache License 2.0. License can be found in LICENSE file.

You might also like...
Implementation of TabTransformer, attention network for tabular data, in Pytorch
Implementation of TabTransformer, attention network for tabular data, in Pytorch

Tab Transformer Implementation of Tab Transformer, attention network for tabular data, in Pytorch. This simple architecture came within a hair's bread

Phil Wang 420 Jan 5, 2023
Boosted neural network for tabular data

XBNet - Xtremely Boosted Network Boosted neural network for tabular data XBNet is an open source project which is built with PyTorch which tries to co

Tushar Sarkar 175 Jan 4, 2023
The official PyTorch implementation of recent paper - SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training
The official PyTorch implementation of recent paper - SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training

This repository is the official PyTorch implementation of SAINT. Find the paper on arxiv SAINT: Improved Neural Networks for Tabular Data via Row Atte

Gowthami Somepalli 284 Dec 21, 2022
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

null 63 Oct 17, 2022
The official implementation of the paper,
The official implementation of the paper, "SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning"

SubTab: Author: Talip Ucar ([email protected]) The official implementation of the paper, SubTab: Subsetting Features of Tabular Data for Self-Supervis

AstraZeneca 98 Dec 29, 2022
A framework for attentive explainable deep learning on tabular data

🧠 kendrite A framework for attentive explainable deep learning on tabular data πŸ’¨ Quick start kedro run 🧱 Built upon Technology Description Links ke

Marnix Koops 3 Nov 6, 2021
A template repository for submitting a job to the Slurm Cluster installed at the DISI - University of Bologna

Cluster di HPC con GPU per esperimenti di calcolo (draft version 1.0) Per poter utilizzare il cluster il primo passo Γ¨ abilitare l'account istituziona

null 20 Dec 16, 2022
Job Assignment System by Real-time Emotion Detection
Job Assignment System by Real-time Emotion Detection

Emotion-Detection Job Assignment System by Real-time Emotion Detection Emotion is the essential role of facial expression and it could provide a lot o

null 1 Feb 8, 2022
Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.
Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.

PS-SC GAN This repository contains the main code for training a PS-SC GAN (a GAN implemented with the Perceptual Simplicity and Spatial Constriction c

Xinqi/Steven Zhu 40 Dec 16, 2022
Comments
  • μ•ˆλ…•ν•˜μ„Έμš” μ—λŸ¬ 문의 λ“œλ¦½λ‹ˆλ‹€.

    μ•ˆλ…•ν•˜μ„Έμš” μ—λŸ¬ 문의 λ“œλ¦½λ‹ˆλ‹€.

    μ΄μ „μ˜ μ—λŸ¬λŠ” ν•΄κ²°ν–ˆλŠ”λ°, μƒˆλ‘œμš΄ μ—λŸ¬λ₯Ό ν•΄κ²°ν•˜μ§€ λͺ»ν•˜κ³  μžˆμŠ΅λ‹ˆλ‹€.

    μ „μ²˜λ¦¬ 뢀뢄인 preprocess.sh νŒŒμΌμ—μ„œ transforemer_2_8_0 의 첫뢀뢄인

    python preprocessing/preprocess.py res/train/KNOW_2017.csv res/test/KNOW_2017_test.csv --data_type=2017 --use_simcse_embeddings

    λ₯Ό μ‹€ν–‰ν•˜λ©΄

    Traceback (most recent call last): File "preprocessing/preprocess.py", line 199, in main(parser.parse_args()) File "preprocessing/preprocess.py", line 93, in main simcse_embeddings = get_text_embeddings_from_simcse( File "C:\Users\rlaal\anaconda3\envs\transformers_2_8_0\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "D:\python\Job-Recommend-Competition-main\preprocessing\text_embedding.py", line 28, in get_text_embeddings_from_simcse from data.dataloader import convert_to_tensor, example_model_setting File "D:\python\Job-Recommend-Competition-main\preprocessing\data\dataloader.py", line 11, in from KoBERT.kobert.utils import get_tokenizer ModuleNotFoundError: No module named 'KoBERT'

    λ‹€μŒκ³Ό 같은 μ—λŸ¬κ°€ λœΉλ‹ˆλ‹€.
    Job-Recommend-Competition-main\preprocessing\data 이 κ²½λ‘œμ— KoBERT 폴더λ₯Ό λ³΅μ‚¬ν•΄μ„œ 넣어봐도 μ—λŸ¬κ°€ ν•΄κ²°λ˜μ§€ μ•Šλ„€μš” 이 경우 μ–΄λ–»κ²Œ ν•΄κ²°ν•˜λŠ”μ§€ 도움을 받을 수 μžˆμ„κΉŒμš”?

    opened by rlaalstn1504 2
Owner
Jungwoo Park
Jungwoo Park
GitHub https://dacon.io/competitions/official/235865/leaderboard
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Deep Cognition and Language Research (DeCLaRe) Lab 89 Dec 26, 2022
Code for NAACL 2021 full paper "Efficient Attentions for Long Document Summarization"

LongDocSum Code for NAACL 2021 paper "Efficient Attentions for Long Document Summarization" This repository contains data and models needed to reprodu

null 56 Jan 2, 2023
This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

[CVPRW 2021] - Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation

Anirudh S Chakravarthy 6 May 3, 2022
[CVPRW 2022] Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network

Attention Helps CNN See Better: Hybrid Image Quality Assessment Network [CVPRW 2022] Code for Hybrid Image Quality Assessment Network [paper] [code] T

IIGROUP 49 Dec 11, 2022
FinGAT: A Financial Graph Attention Networkto Recommend Top-K Profitable Stocks

FinGAT: A Financial Graph Attention Networkto Recommend Top-K Profitable Stocks This is our implementation for the paper: FinGAT: A Financial Graph At

Yu-Che Tsai 64 Dec 13, 2022
[CVPR'21] Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild

IVOS-W Paper Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild Zhaoyun Yin, Jia Zheng, Weixin Luo, Shenhan Qian, Hanli

SVIP Lab 38 Dec 12, 2022
arxiv-sanity, but very lite, simply providing the core value proposition of the ability to tag arxiv papers of interest and have the program recommend similar papers.

arxiv-sanity, but very lite, simply providing the core value proposition of the ability to tag arxiv papers of interest and have the program recommend similar papers.

Andrej 671 Dec 31, 2022
Data and Code for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning"

Introduction Code and data for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning". We cons

Pan Lu 81 Dec 27, 2022
Team nan solution repository for FPT data-centric competition. Data augmentation, Albumentation, Mosaic, Visualization, KNN application

FPT_data_centric_competition - Team nan solution repository for FPT data-centric competition. Data augmentation, Albumentation, Mosaic, Visualization, KNN application

Pham Viet Hoang (Harry) 2 Oct 30, 2022
A standard framework for modelling Deep Learning Models for tabular data

PyTorch Tabular aims to make Deep Learning with Tabular data easy and accessible to real-world cases and research alike.

null 801 Jan 8, 2023