đź’› Code and Dataset for our EMNLP 2021 paper: "Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes"

Overview

Perspective-taking and Pragmatics for Generating
Empathetic Responses Focused on Emotion Causes

figure

Official PyTorch implementation and EmoCause evaluation set of our EMNLP 2021 paper đź’›
Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes. EMNLP, 2021 [Paper]

  • TL;DR: In order to express deeper empathy in dialogues, we argue that responses should focus on the cause of emotions. Inspired by perspective-taking of humans, we propose a generative emotion estimator (GEE) which can recognize emotion cause words solely based on sentence-level emotion labels without word-level annotations (i.e., weak-supervision). To evaluate our approach, we annotate emotion cause words and release the EmoCause evaluation set. We also propose a pragmatics-based method for generating responses focused on targeted words from the context.

Reference

If you use the materials in this repository as part of any published research, we ask you to cite the following paper:

@inproceedings{Kim:2021:empathy,
  title={Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes},
  author={Kim, Hyunwoo and Kim, Byeongchang and Kim, Gunhee},
  booktitle={EMNLP},
  year=2021
}

Implementation

System Requirements

  • Python 3.7.9
  • Pytorch 1.6.0
  • CUDA 10.2 supported GPU with at least 24GB memory
  • See environment.yml for details

Environment setup

Our code is built on the ParlAI framework. We recommend you create a conda environment as follows

conda env create -f environment.yml

and activate it with

conda activate focused-empathy
python -m spacy download en

EmoCause evaluation set for weakly-supervised emotion cause recognition

EmoCause is a dataset of annotated emotion cause words in emotional situations from the EmpatheticDialogues valid and test set. The goal is to recognize emotion cause words in sentences by training only on sentence-level emotion labels without word-level labels (i.e., weakly-supervised emotion cause recognition). EmoCause is based on the fact that humans do not recognize the cause of emotions with supervised learning on word-level cause labels. Thus, we do not provide a training set.

figure

You can download the EmoCause eval set [here].
Note, the dataset will be downloaded automatically when you run the experiment command below.

Data statistics and structure

#Emotion Label type #Label/Utterance #Utterance
EmoCause 32 Word 2.3 4.6K
{
  "original_situation": the original situations in the EmpatheticDialogues,
  "tokenized_situation": tokenized situation utterances using spacy,
  "emotion": emotion labels,
  "conv_id": id for each corresponding conversation in EmpatheticDialogues,
  "annotation": list of tuples: (emotion cause word, index),
  "labels": list of strings containing the emotion cause words
}

Running Experiments

All corresponding models will be downloaded automatically when running the following commands.
We also provide manual download links: [GEE] [Finetuned Blender]

Weakly-supervised emotion cause word recognition with GEE on EmoCause

You can evaluate our proposed Generative Emotion Estimator (GEE) on the EmoCause eval set.

python eval_emocause.py --model agents.gee_agent:GeeCauseInferenceAgent --fp16 False

Focused empathetic response generation with finetuned Blender on EmpatheticDialogues

You can evaluate our approach for generating focused empathetic responses on top of a finetuned Blender (Not familiar with Blender? See here!).

python eval_empatheticdialogues.py --model agents.empathetic_gee_blender:EmpatheticBlenderAgent --model_file data/models/finetuned_blender90m/model --fp16 False --empathy-score False

Adding the --alpha 0 flag will run the Blender without pragmatics. You can also try the random distractor (Plain S1) by adding --distractor-type random.

?? To measure the Interpretation and Exploration scores also, set the --empathy-score to True. It will automatically download the RoBERTa models finetuned on EmpatheticDialogues. For more details on empathy scores, visit the original repo.

Acknowledgements

We thank the anonymous reviewers for their helpful comments on this work.

This research was supported by Samsung Research Funding Center of Samsung Electronics under project number SRFCIT210101. The compute resource and human study are supported by Brain Research Program by National Research Foundation of Korea (NRF) (2017M3C7A1047860).

Have any question?

Please contact Hyunwoo Kim at hyunw.kim at vl dot snu dot ac dot kr.

License

This repository is MIT licensed. See the LICENSE file for details.

Comments
  • AssertionError: Checksum for gee_v1.tar.gz

    AssertionError: Checksum for gee_v1.tar.gz

    I was trying to run the "eval_emocause.py", however, I got the following error:

    AssertionError: Checksum for gee_v1.tar.gz from 1TEKp3YRowAZju4UPXOSufqzU6j6_Z4wy does not match the expected checksum: 06d9b640bd7fa92528c6f3f4cd7715db593a0c1d3284151ba7f615f0d9c4989c != af0584e3c376dd364af0bbc122fea7303b9af52556f4dcbb0fbe8d6a136c0b2a

    I followed all the instruction, including the environment creating. Not sure where it went wrong.

    Btw, I'm really new to the ParlAI. Would you mind to give me some simple instructions for getting the emotional words with GEE? For example, can I create the agent by using "DisplayModel.main"? How to use the "reason_emotion_causes" function after creating the agent?

    I am looking forward to your reply, many thanks!

    opened by jeremychang8 5
  • The version of ParlAI?

    The version of ParlAI?

    Dear authors,

    Thank you for your amazing work. However, some problems occurred when I tried to use your code: `Job is running on ps Traceback (most recent call last): File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 852, in add_task_args agent.add_cmdline_args(self, partial) │ │ └ {'init_opt': None, 'allow_missing_init_opts': False, 'task': 'tasks.emocause', 'download_path': None, 'loglevel': 'info', 'datat... │ └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... └ <class 'tasks.emocause.DefaultTeacher'> TypeError: add_cmdline_args() takes 2 positional arguments but 3 were given

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last): File "eval_emocause.py", line 30, in opt = parser.parse_args() └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 1163, in parse_args self.add_extra_args(args) │ └ None └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 940, in add_extra_args self.add_task_args(task, partial) │ │ └ {'init_opt': None, 'allow_missing_init_opts': False, 'task': 'tasks.emocause', 'download_path': None, 'loglevel': 'info', 'datat... │ └ 'tasks.emocause' └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 859, in add_task_args ) from typ TypeError: Task 'tasks.emocause' appears to have signature add_cmdline_args(argparser) but we have updated the signature to add_cmdline_args(argparser, partial_opt). For details, see https://github.com/facebookresearch/ParlAI/pull/3328.`

    It seems like the ParlAI had updated some signatures for 'add_cmdline_args', as can be seen at: image

    So, my question is what version of ParlAI did your work use? I try to use the latest version and v 1.0.0, it still didn't work.

    Best regard

    opened by RenzeLou 2
  • How can I see predicted predicted emotion class when I evaluate Blender + Focused S1 to ED dataset?

    How can I see predicted predicted emotion class when I evaluate Blender + Focused S1 to ED dataset?

    When I evaluate Blender + Focused S1, I can see this predicted information from that model. How can I see predicted emotion class? I can check emotional words from GEE now.

    {'id': 'EmpatheticBlender', 'episode_done': False, 'text': 'that is horrible . i am so sorry to hear that .', 'token_losses': [('did', 3.5072498321533203), ('you', 2.192622184753418), ('suffer', 6.697482109069824), ('any', 0.485431969165802), ('injuries', 1.560391 902923584), ('?', 0.4341753125190735), ('end', 0.1666361540555954), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0. 0), ('null', 0.0), ('null', 0.0)], 'emotional_words': ['horrifying', 'ran', 'barrels', 'fault', 'survived'], 'metrics': {'loss': AverageMetric(2.149), 'ppl': PPLMetric(8.577), 'token_acc': AverageM etric(0.5714)}

    opened by kimwongyuda 1
  •  Training Code

    Training Code

    Hello, is it possible that you share your code for training both gee model and empathetic dialog? I want to use your model as a baseline for comparison but our school's machines do not have cuda 10.2 (and therefore loading pretrained gee is failing now). I want retrain your model on newer packages (such as newest parlai, torch 1.9, cuda 11 etc) for further use. If there's no training code as well as training instruction, I'm afraid that the comparison will not be fair.

    opened by iammia0o 0
  • Question on EPITOME (Emotional Reaction)

    Question on EPITOME (Emotional Reaction)

    Hello,

    First off, thank you for releasing and maintaining your work, it's very clean and things are working nicely, I know it's a lot of work. I was just curious, my understanding is that EPITOME also has a Emotional Reaction dimension to empathy along with Exploration and Interpretation. Is there a reason that you only use the latter two? Do you have a pre-trained Emotional Reaction model by any chance?

    Thank you!

    opened by ajyl 0
Owner
Hyunwoo Kim
PhD student at Seoul National University CSE
Hyunwoo Kim
Code for our EMNLP 2021 paper “Heterogeneous Graph Neural Networks for Keyphrase Generation”

GATER This repository contains the code for our EMNLP 2021 paper “Heterogeneous Graph Neural Networks for Keyphrase Generation”. Our implementation is

Jiacheng Ye 12 Nov 24, 2022
Code for our paper Aspect Sentiment Quad Prediction as Paraphrase Generation in EMNLP 2021.

Aspect Sentiment Quad Prediction (ASQP) This repo contains the annotated data and code for our paper Aspect Sentiment Quad Prediction as Paraphrase Ge

Isaac 39 Dec 11, 2022
Related resources for our EMNLP 2021 paper

Plan-then-Generate: Controlled Data-to-Text Generation via Planning Authors: Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, and Nigel Collier Code

Yixuan Su 61 Jan 3, 2023
Abstractive opinion summarization system (SelSum) and the largest dataset of Amazon product summaries (AmaSum). EMNLP 2021 conference paper.

Learning Opinion Summarizers by Selecting Informative Reviews This repository contains the codebase and the dataset for the corresponding EMNLP 2021

Arthur BraĹľinskas 39 Jan 1, 2023
Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Loop Story Generation"

Storium GPT-2 Models This is the official repository for the GPT-2 models described in the EMNLP 2020 paper [STORIUM: A Dataset and Evaluation Platfor

Nader Akoury 27 Dec 20, 2022
Codes for our paper "SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge" (EMNLP 2020)

SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge Introduction SentiLARE is a sentiment-aware pre-trained language

null 74 Dec 30, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
EMNLP 2021 Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections Ruiqi Zhong, Kristy Lee*, Zheng Zhang*, Dan Klein EMN

Ruiqi Zhong 42 Nov 3, 2022
Code and data for the EMNLP 2021 paper "Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts". Coming soon!

ToxiChat Code and data for the EMNLP 2021 paper "Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts". Install depen

Ashutosh Baheti 11 Jan 1, 2023
This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories This repo is the code release of EMNLP 2021 con

null 12 Nov 22, 2022
The code repository for EMNLP 2021 paper "Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization".

Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization [Paper] accepted at the EMNLP 2021: Vision Guided Genera

CAiRE 42 Jan 7, 2023
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Deep Cognition and Language Research (DeCLaRe) Lab 89 Dec 26, 2022
Code for EMNLP 2021 paper Contrastive Out-of-Distribution Detection for Pretrained Transformers.

Contra-OOD Code for EMNLP 2021 paper Contrastive Out-of-Distribution Detection for Pretrained Transformers. Requirements PyTorch Transformers datasets

Wenxuan Zhou 27 Oct 28, 2022
Code for EMNLP 2021 main conference paper "Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification"

Text-AutoAugment (TAA) This repository contains the code for our paper Text AutoAugment: Learning Compositional Augmentation Policy for Text Classific

LancoPKU 105 Jan 3, 2023
PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Don’t be Contradicted with Anything!CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System This repository contains the PyTorch im

Libo Qin 25 Sep 6, 2022
PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Libo Qin 12 Sep 26, 2021
This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Towards Persona-Based Empathetic Conversational Models (PEC) This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (E

Zhong Peixiang 35 Nov 17, 2022
The dataset and source code for our paper: "Did You Ask a Good Question? A Cross-Domain Question IntentionClassification Benchmark for Text-to-SQL"

TriageSQL The dataset and source code for our paper: "Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text

Yusen Zhang 22 Nov 9, 2022
Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"

Dataset Distillation by Matching Training Trajectories Project Page | Paper This repo contains code for training expert trajectories and distilling sy

George Cazenavette 256 Jan 5, 2023