Code and dataset for the EMNLP 2021 Finding paper "Can NLI Models Verify QA Systems’ Predictions?"

Overview

Can NLI Models Verify QA Systems' Predictions?

This repository contains the data and code for the following paper:

**Can NLI Models Verify QA Systems' Predictions? **
Jifan Chen, Eunsol Choi, Greg Durrett
EMNLP 2021 Findings

@article{chen2021can,
  title={Can NLI Models Verify QA Systems' Predictions?},
  author={Chen, Jifan and Choi, Eunsol and Durrett, Greg},
  journal={EMNLP Findings},
  year={2021}
}

Datasets

The NLI data converted from QA datasets through our pipeline described in the paper can be found here

Data Format

The data files are formatted as jsonlines; each example is described as the following:

Field Description
example_id Example ID
title_text Title of the Wikipedia page of the example, could be NONE
paragraph_text Paragraph containing the answer
question_text Question
answer_text Answer of the question
answer_sent_text Sentence containing the answer
decontext_answer_sent_text Decontextualized sentence containing the answer
question_statement_text Declarative version of the question by combining the answer
answer_scores Top 5 Answer score computed by the QA(BERT-joint) model
is_correct Whether the answer is correct
answer_sent_text Sentence containing the answer

Models

Getting started

git clone https://github.com/jifan-chen/QA-Verification-Via-NLI.git

Install the dependencies by running pip install -r requirements.txt

Question Converter & Decontextualizer

See README in seq2seq_converter.

NQ-NLI

coming soon

Contact

Please contact at [email protected] if you have any questions.

Issues
  • ModuleNotFoundError

    ModuleNotFoundError

    Hello,

    I am running your code as it is on a WSL Ubuntu.

    I am getting this error ModuleNotFoundError: No module named 'seq2seq_converter.utils'; 'seq2seq_converter' is not a package

    You should move the *.sh files outside of the seq2seq_converter folder.

    opened by ChadiHelwe 0
Owner
Jifan Chen
Stay young, stay simple
Jifan Chen
A Survey of Natural Language Generation in Task-Oriented Dialogue System (TOD): Recent Advances and New Frontiers

A Survey of Natural Language Generation in Task-Oriented Dialogue System (TOD): Recent Advances and New Frontiers

Libo Qin 86 Oct 12, 2021
A Paper List for Speech Translation

Keyword: Speech Translation, Spoken Language Processing, Natural Language Processing

null 89 Oct 21, 2021
Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

Udit Arora 15 Oct 7, 2021
Must-read papers on improving efficiency for pre-trained language models.

Must-read papers on improving efficiency for pre-trained language models.

Tobias Lee 44 Oct 15, 2021
COVID-19 Related NLP Papers

COVID-19 outbreak has become a global pandemic. NLP researchers are fighting the epidemic in their own way.

xcfeng 21 Oct 19, 2021
Beautiful visualizations of how language differs among document types.

Scattertext 0.1.0.0 A tool for finding distinguishing terms in corpora and displaying them in an interactive HTML scatter plot. Points corresponding t

Jason S. Kessler 1.7k Oct 23, 2021
Beautiful visualizations of how language differs among document types.

Scattertext 0.1.0.0 A tool for finding distinguishing terms in corpora and displaying them in an interactive HTML scatter plot. Points corresponding t

Jason S. Kessler 1.5k Feb 17, 2021
基于pytorch+bert的中文事件抽取

pytorch_bert_event_extraction 基于pytorch+bert的中文事件抽取,主要思想是QA(问答)。 要预先下载好chinese-roberta-wwm-ext模型,并在运行时指定模型的位置。

西西嘛呦 7 Oct 13, 2021
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 52.8k Oct 21, 2021
A framework for evaluating Knowledge Graph Embedding Models in a fine-grained manner.

A framework for evaluating Knowledge Graph Embedding Models in a fine-grained manner.

NEC Laboratories Europe 7 Oct 14, 2021
Task-based datasets, preprocessing, and evaluation for sequence models.

SeqIO: Task-based datasets, preprocessing, and evaluation for sequence models. SeqIO is a library for processing sequential data to be fed into downst

Google 78 Oct 18, 2021
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP

TextAttack ?? Generating adversarial examples for NLP models [TextAttack Documentation on ReadTheDocs] About • Setup • Usage • Design About TextAttack

QData 1.7k Oct 23, 2021
Conditional probing: measuring usable information beyond a baseline

Conditional probing: measuring usable information beyond a baseline

John Hewitt 10 Oct 19, 2021
💛 Code and Dataset for our EMNLP 2021 paper: "Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes"

Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes Official PyTorch implementation and EmoCause evaluatio

Hyunwoo Kim 15 Oct 11, 2021
Code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

Chenhe Dong 16 Oct 11, 2021
Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Amazon Web Services - Labs 1000 Apr 19, 2021
Open-Source Toolkit for End-to-End Speech Recognition leveraging PyTorch-Lightning and Hydra.

?? Contributing to OpenSpeech ?? OpenSpeech provides reference implementations of various ASR modeling papers and three languages recipe to perform ta

Openspeech TEAM 273 Oct 17, 2021