💛 Code and Dataset for our EMNLP 2021 paper: "Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes"

Overview

Perspective-taking and Pragmatics for Generating
Empathetic Responses Focused on Emotion Causes

figure

Official PyTorch implementation and EmoCause evaluation set of our EMNLP 2021 paper 💛
Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes. EMNLP, 2021 [Paper coming soon!]

  • TL;DR: In order to express deeper empathy in dialogues, we argue that responses should focus on the cause of emotions. Inspired by perspective-taking of humans, we propose a generative emotion estimator (GEE) which can recognize emotion cause words solely based on sentence-level emotion labels without word-level annotations (i.e., weak-supervision). To evaluate our approach, we annotate emotion cause words and release the EmoCause evaluation set. We also propose a pragmatics-based method for generating responses focused on targeted words from the context.

Reference

If you use the materials in this repository as part of any published research, we ask you to cite the following paper:

@inproceedings{Kim:2021:empathy,
  title={Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes},
  author={Kim, Hyunwoo and Kim, Byeongchang and Kim, Gunhee},
  booktitle={EMNLP},
  year=2021
}

Implementation

System Requirements

  • Python 3.7.9
  • Pytorch 1.6.0
  • CUDA 10.2 supported GPU with at least 24GB memory
  • See environment.yml for details

Environment setup

Our code is built on the ParlAI framework. We recommend you create a conda environment as follows

conda env create -f environment.yml

and activate it with

conda activate focused-empathy
python -m spacey download en

EmoCause evaluation set for weakly-supervised emotion cause recognition

EmoCause is a dataset of annotated emotion cause words in emotional situations from the EmpatheticDialogues valid and test set. The goal is to recognize emotion cause words in sentences by training only on sentence-level emotion labels without word-level labels (i.e., weakly-supervised emotion cause recognition). EmoCause is based on the fact that humans do not recognize the cause of emotions with supervised learning on word-level cause labels. Thus, we do not provide a training set.

figure

You can download the EmoCause eval set [here].
Note, the dataset will be downloaded automatically when you run the experiment command below.

Data statistics and structure

#Emotion Label type #Label/Utterance #Utterance
EmoCause 32 Word 2.3 4.6K
{
  "original_situation": the original situations in the EmpatheticDialogues,
  "tokenized_situation": tokenized situation utterances using spacy,
  "emotion": emotion labels,
  "conv_id": id for each corresponding conversation in EmpatheticDialogues,
  "annotation": list of tuples: (emotion cause word, index),
  "labels": list of strings containing the emotion cause words
}

Running Experiments

All corresponding models will be downloaded automatically when running the following commands.
We also provide manual download links: [GEE] [Finetuned Blender]

Weakly-supervised emotion cause word recognition with GEE on EmoCause

You can evaluate our proposed Generative Emotion Estimator (GEE) on the EmoCause eval set.

python eval_emocause.py --model agents.gee_agent:GeeCauseInferenceAgent --fp16 False

Focused empathetic response generation with finetuned Blender on EmpatheticDialogues

You can evaluate our approach for generating focused empathetic responses on top of a finetuned Blender (Not familiar with Blender? See here!).

python eval_empatheticdialogues.py --model agents.empathetic_gee_blender:EmpatheticBlenderAgent --model_file data/models/finetuned_blender90m/model --fp16 False --empathy-score False

Adding the --alpha 0 flag will run the Blender without pragmatics. You can also try the random distractor (Plain S1) by adding --distractor-type random.

💡 To measure the Interpretation and Exploration scores also, set the --empathy-score to True. It will automatically download the RoBERTa models finetuned on EmpatheticDialogues. For more details on empathy scores, visit the original repo.

Acknowledgements

We thank the anonymous reviewers for their helpful comments on this work.

This research was supported by Samsung Research Funding Center of Samsung Electronics under project number SRFCIT210101. The compute resource and human study are supported by Brain Research Program by National Research Foundation of Korea (NRF) (2017M3C7A1047860).

Have any question?

Please contact Hyunwoo Kim at hyunw.kim at vl dot snu dot ac dot kr.

License

This repository is MIT licensed. See the LICENSE file for details.

Comments
  • AssertionError: Checksum for gee_v1.tar.gz

    AssertionError: Checksum for gee_v1.tar.gz

    I was trying to run the "eval_emocause.py", however, I got the following error:

    AssertionError: Checksum for gee_v1.tar.gz from 1TEKp3YRowAZju4UPXOSufqzU6j6_Z4wy does not match the expected checksum: 06d9b640bd7fa92528c6f3f4cd7715db593a0c1d3284151ba7f615f0d9c4989c != af0584e3c376dd364af0bbc122fea7303b9af52556f4dcbb0fbe8d6a136c0b2a

    I followed all the instruction, including the environment creating. Not sure where it went wrong.

    Btw, I'm really new to the ParlAI. Would you mind to give me some simple instructions for getting the emotional words with GEE? For example, can I create the agent by using "DisplayModel.main"? How to use the "reason_emotion_causes" function after creating the agent?

    I am looking forward to your reply, many thanks!

    opened by jeremychang8 5
  • The version of ParlAI?

    The version of ParlAI?

    Dear authors,

    Thank you for your amazing work. However, some problems occurred when I tried to use your code: `Job is running on ps Traceback (most recent call last): File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 852, in add_task_args agent.add_cmdline_args(self, partial) │ │ └ {'init_opt': None, 'allow_missing_init_opts': False, 'task': 'tasks.emocause', 'download_path': None, 'loglevel': 'info', 'datat... │ └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... └ <class 'tasks.emocause.DefaultTeacher'> TypeError: add_cmdline_args() takes 2 positional arguments but 3 were given

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last): File "eval_emocause.py", line 30, in opt = parser.parse_args() └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 1163, in parse_args self.add_extra_args(args) │ └ None └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 940, in add_extra_args self.add_task_args(task, partial) │ │ └ {'init_opt': None, 'allow_missing_init_opts': False, 'task': 'tasks.emocause', 'download_path': None, 'loglevel': 'info', 'datat... │ └ 'tasks.emocause' └ ParlaiParser(prog='eval_emocause.py', usage=None, description='Evaluate a model', formatter_class=<class 'parlai.core.params.Cus... File "/data/lourenze/ECD/baseline/focused-empathy/parlai/core/params.py", line 859, in add_task_args ) from typ TypeError: Task 'tasks.emocause' appears to have signature add_cmdline_args(argparser) but we have updated the signature to add_cmdline_args(argparser, partial_opt). For details, see https://github.com/facebookresearch/ParlAI/pull/3328.`

    It seems like the ParlAI had updated some signatures for 'add_cmdline_args', as can be seen at: image

    So, my question is what version of ParlAI did your work use? I try to use the latest version and v 1.0.0, it still didn't work.

    Best regard

    opened by RenzeLou 2
  • How can I see predicted predicted emotion class when I evaluate Blender + Focused S1 to ED dataset?

    How can I see predicted predicted emotion class when I evaluate Blender + Focused S1 to ED dataset?

    When I evaluate Blender + Focused S1, I can see this predicted information from that model. How can I see predicted emotion class? I can check emotional words from GEE now.

    {'id': 'EmpatheticBlender', 'episode_done': False, 'text': 'that is horrible . i am so sorry to hear that .', 'token_losses': [('did', 3.5072498321533203), ('you', 2.192622184753418), ('suffer', 6.697482109069824), ('any', 0.485431969165802), ('injuries', 1.560391 902923584), ('?', 0.4341753125190735), ('end', 0.1666361540555954), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0.0), ('null', 0. 0), ('null', 0.0), ('null', 0.0)], 'emotional_words': ['horrifying', 'ran', 'barrels', 'fault', 'survived'], 'metrics': {'loss': AverageMetric(2.149), 'ppl': PPLMetric(8.577), 'token_acc': AverageM etric(0.5714)}

    opened by kimwongyuda 1
  •  Training Code

    Training Code

    Hello, is it possible that you share your code for training both gee model and empathetic dialog? I want to use your model as a baseline for comparison but our school's machines do not have cuda 10.2 (and therefore loading pretrained gee is failing now). I want retrain your model on newer packages (such as newest parlai, torch 1.9, cuda 11 etc) for further use. If there's no training code as well as training instruction, I'm afraid that the comparison will not be fair.

    opened by iammia0o 0
  • Question on EPITOME (Emotional Reaction)

    Question on EPITOME (Emotional Reaction)

    Hello,

    First off, thank you for releasing and maintaining your work, it's very clean and things are working nicely, I know it's a lot of work. I was just curious, my understanding is that EPITOME also has a Emotional Reaction dimension to empathy along with Exploration and Interpretation. Is there a reason that you only use the latter two? Do you have a pre-trained Emotional Reaction model by any chance?

    Thank you!

    opened by ajyl 0
Owner
Hyunwoo Kim
PhD student at Seoul National University CSE
Hyunwoo Kim
This is the code for the EMNLP 2021 paper AEDA: An Easier Data Augmentation Technique for Text Classification

The baseline code is for EDA: Easy Data Augmentation techniques for boosting performance on text classification tasks

Akbar Karimi 81 Dec 9, 2022
This repository contains the code for EMNLP-2021 paper "Word-Level Coreference Resolution"

Word-Level Coreference Resolution This is a repository with the code to reproduce the experiments described in the paper of the same name, which was a

null 79 Dec 27, 2022
Code for EMNLP 2021 main conference paper "Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification"

Code for EMNLP 2021 main conference paper "Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification"

LancoPKU 105 Jan 3, 2023
Code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

Chenhe Dong 28 Nov 10, 2022
Code for Findings at EMNLP 2021 paper: "Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning"

Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning This repo is for Findings at EMNLP 2021 paper: Learn Cont

INK Lab @ USC 6 Sep 2, 2022
Code to reproduce the results of the paper 'Towards Realistic Few-Shot Relation Extraction' (EMNLP 2021)

Realistic Few-Shot Relation Extraction This repository contains code to reproduce the results in the paper "Towards Realistic Few-Shot Relation Extrac

Bloomberg 8 Nov 9, 2022
Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

Udit Arora 19 Oct 28, 2022
Code for our paper "Mask-Align: Self-Supervised Neural Word Alignment" in ACL 2021

Mask-Align: Self-Supervised Neural Word Alignment This is the implementation of our work Mask-Align: Self-Supervised Neural Word Alignment. @inproceed

THUNLP-MT 46 Dec 15, 2022
Code for our ACL 2021 paper - ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

ConSERT Code for our ACL 2021 paper - ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer Requirements torch==1.6.0

Yan Yuanmeng 478 Dec 25, 2022
Code for our ACL 2021 (Findings) Paper - Fingerprinting Fine-tuned Language Models in the wild .

?? Fingerprinting Fine-tuned Language Models in the wild This is the code and dataset for our ACL 2021 (Findings) Paper - Fingerprinting Fine-tuned La

LCS2-IIITDelhi 5 Sep 13, 2022
Code for our paper "Transfer Learning for Sequence Generation: from Single-source to Multi-source" in ACL 2021.

TRICE: a task-agnostic transferring framework for multi-source sequence generation This is the source code of our work Transfer Learning for Sequence

THUNLP-MT 9 Jun 27, 2022
The FinQA dataset from paper: FinQA: A Dataset of Numerical Reasoning over Financial Data

Data and code for EMNLP 2021 paper "FinQA: A Dataset of Numerical Reasoning over Financial Data"

Zhiyu Chen 114 Dec 29, 2022
🍊 PAUSE (Positive and Annealed Unlabeled Sentence Embedding), accepted by EMNLP'2021 🌴

PAUSE: Positive and Annealed Unlabeled Sentence Embedding Sentence embedding refers to a set of effective and versatile techniques for converting raw

EQT 21 Dec 15, 2022
[EMNLP 2021] LM-Critic: Language Models for Unsupervised Grammatical Error Correction

LM-Critic: Language Models for Unsupervised Grammatical Error Correction This repo provides the source code & data of our paper: LM-Critic: Language M

Michihiro Yasunaga 98 Nov 24, 2022
IndoBERTweet is the first large-scale pretrained model for Indonesian Twitter. Published at EMNLP 2021 (main conference)

IndoBERTweet ?? ???? 1. Paper Fajri Koto, Jey Han Lau, and Timothy Baldwin. IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effe

IndoLEM 40 Nov 30, 2022
Beta Distribution Guided Aspect-aware Graph for Aspect Category Sentiment Analysis with Affective Knowledge. Proceedings of EMNLP 2021

AAGCN-ACSA EMNLP 2021 Introduction This repository was used in our paper: Beta Distribution Guided Aspect-aware Graph for Aspect Category Sentiment An

Akuchi 36 Dec 18, 2022
EMNLP'2021: Can Language Models be Biomedical Knowledge Bases?

BioLAMA BioLAMA is biomedical factual knowledge triples for probing biomedical LMs. The triples are collected and pre-processed from three sources: CT

DMIS Laboratory - Korea University 41 Nov 18, 2022
A2T: Towards Improving Adversarial Training of NLP Models (EMNLP 2021 Findings)

A2T: Towards Improving Adversarial Training of NLP Models This is the source code for the EMNLP 2021 (Findings) paper "Towards Improving Adversarial T

QData 17 Oct 15, 2022
Official code of our work, Unified Pre-training for Program Understanding and Generation [NAACL 2021].

PLBART Code pre-release of our work, Unified Pre-training for Program Understanding and Generation accepted at NAACL 2021. Note. A detailed documentat

Wasi Ahmad 138 Dec 30, 2022