Implementation of the paper 'Sentence Bottleneck Autoencoders from Transformer Language Models'

Related tags

Third-party APIs Wrappers autobot
Overview

Introduction

This repository contains the code for the paper Sentence Bottleneck Autoencoders from Transformer Language Models by Ivan Montero, Nikolaos Pappas, and Noah A. Smith published at EMNLP 2021.

This paper proposes an approach to learning sentence representations by applying an autoencoder on top of pretrained masked LMs.

Further documentation and code cleanup are under the works!

Installation

Coming soon!

This code was tested under Ubuntu 18.04, Python 3.7, and PyTorch 1.6.

To install, run the setup file.

bash setup.sh

We use modified versions of the following repositories:

transformers==3.3.1
fairseq==0.9.0

Example Usage

Coming soon!

Modified Files

Coming soon!

You might also like...
ANKIT-OS/STYLISH-TEXT is a special repository. Its Is A Telegram Bot Which Can Translate Your Text Into 100+ Language
ANKIT-OS/STYLISH-TEXT is a special repository. Its Is A Telegram Bot Which Can Translate Your Text Into 100+ Language

🔥 ᴳᴼᴼᴳᴸᴱ⁻ᵀᴿᴬᴺᔆᴸᴬᵀᴱᴿ 🔥 The owner would not be responsible for any kind of bans due to the bot. • ⚡ INSTALLING ⚡ • • 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs 🔰 • If

ANKIT KUMAR 1 Dec 23, 2021
Weather telegram bot with aiogram, on Russian language

weather_bot Weather telegram bot with aiogram, on Russian language #RU Бот по определению погоды в Telegram, написана на библиотеке aiogram, весь инте

LinkxWan 0 Jan 6, 2022
A stack-based systems language that supports structures, functions, expressions, and user-defined operator behaviour

A stack-based systems language that supports structures, functions, expressions, and user-defined operator behaviour. Currently compiles to URCL with plans to add additional formats in the future.

Lucida Dragon 3 Nov 3, 2022
WordPress models and views for Django.

django-wordpress Models and views for reading a WordPress database. Compatible with WordPress version 3.5+. django-wordpress is a project of ISL and t

Jeremy Carbaugh 332 Dec 24, 2022
Python API for working with RESQML models

resqpy: Python API for working with RESQML models Introduction resqpy is a pure python package which provides a programming interface (API) for readin

BP 44 Dec 14, 2022
go-cqhttp API typing annoations, return data models and utils for nonebot

go-cqhttp API typing annoations, return data models and utils for nonebot

风屿 6 Jan 4, 2023
The python SDK for Eto, the AI focused data platform for teams bringing AI models to production

Eto Labs Python SDK This is the python SDK for Eto, the AI focused data platform for teams bringing AI models to production. The python SDK makes it e

null 5 Apr 21, 2022
An api, written in Python, for Investopedia's paper trading stock simulator.

investopedia-trading-api An API, written in Python, for Investopedia's paper trading stock simulator. Pull requests welcome. This library is now Pytho

Kirk Thaker 178 Jan 6, 2023
Facebook open graph api implementation using the Django web framework in python

Django Facebook by Thierry Schellenbach (mellowmorning.com) Status Django and Facebook are both rapidly changing at the moment. Meanwhile, I'm caught

Thierry Schellenbach 1.4k Dec 29, 2022
Comments
  • Gating mechanism

    Gating mechanism

    Hello @ivanmontero, interesting work what you did. I was trying to find where part of the code is the gating method you mentioned in section 2.2 (Decoder) of the paper. Could you tell me what script it is in?

    Thank you in advance!

    opened by vgaraujov 1
  • method related files

    method related files

    Hi, I'm interested your implementations, can you point me to method related files, I found it's hard to find it since you change trasformers and fairseq directly. BTW, I think inheritance or override class and function without change them directly is a better way.

    opened by tshu-w 5
Owner
Ivan Montero
UW Undergrad | NLP Research Previously: @Apple, @Google, @Facebook
Ivan Montero
GitHub
CDIoU and CDIoU loss is like a convenient plug-in that can be used in multiple models. CDIoU and CDIoU loss have different excellent performances in several models such as Faster R-CNN, YOLOv4, RetinaNet and . There is a maximum AP improvement of 1.9% and an average AP of 0.8% improvement on MS COCO dataset, compared to traditional evaluation-feedback modules. Here we just use as an example to illustrate the code.

CDIoU-CDIoUloss CDIoU and CDIoU loss is like a convenient plug-in that can be used in multiple models. CDIoU and CDIoU loss have different excellent p

Alan D Chen 24 Jul 19, 2022
Contrastive Language-Audio Pretraining

CLAP Contrastive Language-Audio Pretraining In due time this repo will be full of lovely things, I hope. Feel free to check out the Issues if you're i

Charles Foster 83 Dec 1, 2022
Python library for the DeepL language translation API.

The DeepL API is a language translation API that allows other computer programs to send texts and documents to DeepL's servers and receive high-quality translations. This opens a whole universe of opportunities for developers: any translation product you can imagine can now be built on top of DeepL's best-in-class translation technology.

DeepL 535 Jan 4, 2023
Asynchronous and also synchronous non-official QvaPay client for asyncio and Python language.

Asynchronous and also synchronous non-official QvaPay client for asyncio and Python language. This library is still under development, the interface could be changed.

Leynier Gutiérrez González 8 Sep 18, 2021
An advanced telegram language translator bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License -> https://github.com/FayasNoushad/Translator-Bot-V3/blob/main/LICE

Fayas Noushad 19 Dec 24, 2022
Integrating the Daraja-Api with Python language

Mpesa-Daraja-Api Integrating the Daraja-Api with Python language. Credentials.py file This file contains the consumer key and the consumer secrete key

Morvin Ian 3 Nov 9, 2022
A simple language translator with python and google translate api

Language translator with python A simple language translator with python and google translate api Install pip and python 3.9. All the required depende

null 0 Nov 11, 2021
Cookiecutter templates for Serverless applications using AWS SAM and the Rust programming language.

Cookiecutter SAM template for Lambda functions in Rust This is a Cookiecutter template to create a serverless application based on the Serverless Appl

AWS Samples 24 Nov 11, 2022
A simple telegram bot to recognize lengthy voice files to text and vice versa with multiple language support.

Voicebot A simple Telegram bot to convert lengthy voice clips to text and vice versa with supporting languages. Mandatory Variables API_HASH - Yo

Renjith Mangal 12 Oct 21, 2022
fhempy is a FHEM binding to write modules in Python language

fhempy (BETA) fhempy allows the usage of Python 3 (NOT 2!) language to write FHEM modules. Python 3.7 or higher is required, therefore I recommend usi

Dominik 27 Dec 14, 2022