[EMNLP 2020] Keep CALM and Explore: Language Models for Action Generation in Text-based Games

Related tags

Deep Learning nlp rl gpt n-gram calm text-based-game

Overview

Contextual Action Language Model (CALM) and the ClubFloyd Dataset

Code and data for paper Keep CALM and Explore: Language Models for Action Generation in Text-based Games at EMNLP 2020.

Overview

Our ClubFloyd dataset (calm/lm_data.zip) is crawled from the ClubFloyd website and contains 426 human gameplay transcripts, which cover 590 text-based games of diverse genres and styles.

The data consists of 223,527 context-action pairs in the format [CLS] observation [SEP] action [SEP] next observation [SEP] next action [SEP]. We use [CLS] observation [SEP] action [SEP] next observation [SEP] as the context to train language models (n-gram, GPT-2) to predict next action [SEP], and show that this action generation ability generalizes to unseen games and supports gameplay when combined with reinforcement learning.

Getting Started

Clone repo and install dependencies:

pip install torch==1.4 transformers==2.5.1 jericho fasttext wandb importlib_metadata
git clone https://github.com/princeton-nlp/calm-textgame && cd calm-textgame
ln -s ../lm calm && ln -s ../lm drrn

(If the pip installation fails for fasttext, try the build steps here: https://github.com/facebookresearch/fastText#building-fasttext-for-python)

Train CALM:

cd calm
unzip lm_data.zip
python train.py

Trained model weights can be downloaded here for both GPT-2 and n-gram models.

Then train DRRN using the trained CALM:

cd ../drrn
python train.py --rom_path ../games/${GAME} --lm_path ${PATH_TO_CALM} --lm_type ${gpt_or_ngram}

To quickly try out the GPT-2 CALM model:

from lm import GPT2LM
model = GPT2LM("model_weights/gpt2")
print(model.generate("[CLS] observation [SEP] action [SEP] next observation [SEP]", k=30))

Citation

@inproceedings{yao2020calm,
    title={Keep CALM and Explore: Language Models for Action Generation in Text-based Games},
    author={Yao, Shunyu and Rao, Rohan and Hausknecht, Matthew and Narasimhan, Karthik},
    booktitle={Empirical Methods in Natural Language Processing (EMNLP)},
    year={2020}
}

Acknowledgements

Thanks Jacqueline for hosting the wonderful ClubFloyd website and granting our use!

The code borrows from TDQN (for the RL part) and Huggingface Transformers (for the CALM part).

For any questions please contact Shunyu Yao <[email protected]>.

Comments

Any try on other RL agent ?

Hi, thanks for the great work of text game. I have one question about the RL agent. In this paper, your agent is Deep Reinforcement Relevance Network (DRRN) from ACL2016 paper. I am wondering did you ever conduct some preliminary experiments on more powerful encoding function like BERT for better contextualized word embedding ? Do you have some intuition for making Transformer as Q-network in DRL ? Much Thanks !

opened by Hannibal046 1
Train DRNN without CALM

How can I train the DRNN without using CALM?? And only using the default handicap version of Jericho. Just wanted to regenerate baseline results. Thanks

opened by agSidharth 1
key = hash(tuple(tuple(input_ids), k))

https://github.com/princeton-nlp/calm-textgame/blob/master/lm/gpt.py#L36 Is here an error or intended? tuple seems only accept one parameter. I used Python3.6. Thanks.

opened by zhaozj89 1
Example: text game “A Dark Room”

I have found this text adventure "A Dark Room - A Minimalist Text Adventure"

I would like to tweak this game with CALM. Which would be the right starting point?

Thanks!

opened by loretoparisi 1
Inference example

Thanks for this work! It would be worth to provide an inference example using the provided gpt-2 model weights for a given set of observations.

Thank you in advance.

opened by loretoparisi 1

[EMNLP 2020] Keep CALM and Explore: Language Models for Action Generation in Text-based Games

Related tags

Overview

Contextual Action Language Model (CALM) and the ClubFloyd Dataset

Overview

Getting Started

Citation

Acknowledgements

Comments

Any try on other RL agent ?

Train DRNN without CALM

key = hash(tuple(tuple(input_ids), k))

Example: text game “A Dark Room”

Inference example

Owner

Princeton Natural Language Processing

Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Loop Story Generation"

An open source app to help calm you down when needed.

This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Explore extreme compression for pre-trained language models

The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Codes for our paper "SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge" (EMNLP 2020)

EMNLP 2020 - Summarizing Text on Any Aspects

Related resources for our EMNLP 2021 paper

Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked.

Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Human Action Controller - A human action controller running on different platforms.

EMNLP 2021 Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

Summary Explorer is a tool to visually explore the state-of-the-art in text summarization.

📝 Wrapper library for text generation / language models at char and word level with RNN in TensorFlow

The code repository for EMNLP 2021 paper "Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization".

Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021)

Pytorch implementation of paper "Efficient Nearest Neighbor Language Models" (EMNLP 2021)