Towards Debiasing NLU Models from Unknown Biases

Overview

Towards Debiasing NLU Models from Unknown Biases

Abstract: NLU models often exploit biased features to achieve high dataset-specific performance without properly learning the intended task. Recently proposed debiasing methods are shown to be effective in mitigating this tendency. However, these methods rely on a major assumption that the type of biased features is known a-priori, which limits their application to many NLU tasks and datasets. In this work, we present the first step to bridge this gap by introducing a self-debiasing framework that prevents models from mainly utilizing biases without knowing them in advance. The proposed framework is general and complementary to the existing debiasing methods. We show that the proposed framework allows these existing methods to retain the improvement on the challenge datasets (i.e., sets of examples designed to expose models’ reliance to biases) without specifically targeting certain biases. Furthermore, the evaluation suggests that applying the framework results in improved overall robustness.

The repository contains the code to reproduce our work in debiasing NLU models without prior information on biases. We provide 3 runs of experiment that are shown in our paper:

  1. Debias MNLI model from syntactic bias and evaluate on HANS as the out-of-distribution data using example reweighting.
  2. Debias MNLI model from syntactic bias and evaluate on HANS as the out-of-distribution data using product of expert.
  3. Debias MNLI model from syntactic bias and evaluate on HANS as the out-of-distribution data using confidence regularization.

Requirements

The code requires python >= 3.6 and pytorch >= 1.1.0.

Additional required dependencies can be found in requirements.txt. Install all requirements by running:

pip install -r requirements.txt

Data

Our experiments use MNLI dataset version provided by GLUE benchmark. Download the file from here, and unzip under the directory ./dataset The dataset directory should be structured as the following:

└── dataset 
    └── MNLI
        ├── train.tsv
        ├── dev_matched.tsv
        ├── dev_mismatched.tsv
        ├── dev_mismatched.tsv

Running the experiments

For each evaluation setting, use the --mode arguments to set the appropriate loss function. Choose the annealed version of the loss function for reproducing the annealed results.

To reproduce our result on MNLI ⮕ HANS, run the following:

cd src/
CUDA_VISIBLE_DEVICES=9 python train_distill_bert.py \
  --output_dir ../experiments_self_debias_mnli_seed111/bert_reweighted_sampled2K_teacher_seed111_annealed_1to08 \
  --do_train --do_eval --mode reweight_by_teacher_annealed \
  --custom_teacher ../teacher_preds/mnli_trained_on_sample2K_seed111.json --seed 111 --which_bias hans

Biased examples identification

To obtain predictions of the shallow models, we train the same model architecture on the fraction of the dataset. For MNLI we subsample 2000 examples and train the model for 5 epochs. For obtaining shallow models of other datasets please see the appendix of our paper. The shallow model can be obtained with the command below:

cd src/
CUDA_VISIBLE_DEVICES=9 python train_distill_bert.py \
 --output_dir ../experiments_shallow_mnli/bert_base_sampled2K_seed111 \
 --do_train --do_eval --do_eval_on_train --mode none\
 --seed 111 --which_bias hans --debug --num_train_epochs 5 --debug_num 2000

Once the training and the evaluation on train set is done, copy the probability json files in the output directory to ../teacher_preds/mnli_trained_on_sample2K_seed111.json.

Expected results

Results on the MNLI ⮕ HANS setting without annealing:

Mode Seed MNLI-m MNLI-mm HANS avg.
None 111 84.57 84.72 62.04
reweighting 111 81.8 82.3 72.1
PoE 111 81.5 81.1 70.3
conf-reg 222 83.7 84.1 68.7
You might also like...
This library provides an abstraction to perform Model Versioning using Weight & Biases.
This library provides an abstraction to perform Model Versioning using Weight & Biases.

Description This library provides an abstraction to perform Model Versioning using Weight & Biases. Features Version a new trained model Promote a mod

Neural Factorization of Shape and Reflectance Under An Unknown Illumination
Neural Factorization of Shape and Reflectance Under An Unknown Illumination

NeRFactor [Paper] [Video] [Project] This is the authors' code release for: NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown I

Code release for our paper,
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

CUP-DNN is a deep neural network model used to predict tissues of origin for cancers of unknown of primary.

CUP-DNN CUP-DNN is a deep neural network model used to predict tissues of origin for cancers of unknown of primary. The model was trained on the expre

Multi-robot collaborative exploration and mapping through Voronoi partition and DRL in unknown environment
Multi-robot collaborative exploration and mapping through Voronoi partition and DRL in unknown environment

Voronoi Multi_Robot Collaborate Exploration Introduction In the unknown environment, the cooperative exploration of multiple robots is completed by Vo

This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Towards Persona-Based Empathetic Conversational Models (PEC) This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (E

This is the repository for paper NEEDLE: Towards Non-invertible Backdoor Attack to Deep Learning Models.

This is the repository for paper NEEDLE: Towards Non-invertible Backdoor Attack to Deep Learning Models.

This is the code of NeurIPS'21 paper "Towards Enabling Meta-Learning from Target Models".

ST This is the code of NeurIPS 2021 paper "Towards Enabling Meta-Learning from Target Models". If you use any content of this repo for your work, plea

Official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models.

GLIDE This is the official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing w

Comments
  • No such file or directory: '../dataset/glue_multinli/dev_matched_easy.tsv'?

    No such file or directory: '../dataset/glue_multinli/dev_matched_easy.tsv'?

    hello! I'm puzzled by this problem: No such file or directory: '../dataset/glue_multinli/dev_matched_easy.tsv' Is there any data missing? Looking forward to you reply. Thanks!

    opened by dqxiu 0
  • Reproduction of

    Reproduction of "conf-reg" mode result

    Hi, I have an inquiry about reproduction of "conf-reg" mode result. Refering to the source code, I set the mode input as "--mode smoothed_distill_annealed", but I got the results in MNLI as MNLI-m : 64.9 / MNLI-mm : 66.9 / HANS : 50.0, far behind the performances in the paper. Below are the commands I used. May I ask if there is anything wrong with them? Thank you.

    CUDA_VISIBLE_DEVICES=0 python train_distill_bert.py
    --output_dir ../experiments_shallow_mnli/bert_base_sampled2K_seed111
    --do_train --do_eval --do_eval_on_train --mode none
    --seed 111 --which_bias hans --debug --num_train_epochs 5.0 --debug_num 2000

    (I then moved a resulting file "eval_mnli_train_answers.json" to ../teacher_preds directory.)

    CUDA_VISIBLE_DEVICES=0 python train_distill_bert.py
    --output_dir ../experiments_self_debias_mnli_seed111/bert_smoothed_distill_sampled2K_teacher_seed111_annealed
    --do_train --do_eval --mode smoothed_distill_annealed --num_train_epochs 5.0
    --custom_teacher ../teacher_preds/eval_mnli_train_answers.json --seed 111 --which_bias hans

    opened by jej127 0
  • Release QQP and FEVER code?

    Release QQP and FEVER code?

    Dear @putama, thank you so much for releasing the MNLI debiasing code.

    It would be extremely useful to also have a look at the QQP and FEVER implementations. Would it be possible to release those?

    opened by mariomeissner 0
  • Remaining bias files

    Remaining bias files

    Thank you for releasing this code.

    Could you please also upload the remaining bias files (for FEVER and QQP)? Or point me towards a way to obtain them myself?

    opened by mariomeissner 0
Owner
Ubiquitous Knowledge Processing Lab
Ubiquitous Knowledge Processing Lab
[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

Towards Understanding and Mitigating Social Biases in Language Models This repo contains code and data for evaluating and mitigating bias from generat

Paul Liang 42 Jan 3, 2023
Codes and models for the paper "Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction".

GNN_PPI Codes and models for the paper "Learning Unknown from Correlations: Graph Neural Network for Inter-novel-protein Interaction Prediction". Lear

Ursa Zrimsek 2 Dec 14, 2022
Implementation of "Debiasing Item-to-Item Recommendations With Small Annotated Datasets" (RecSys '20)

Debiasing Item-to-Item Recommendations With Small Annotated Datasets This is the code for our RecSys '20 paper. Other materials can be found here: Ful

Microsoft 34 Aug 10, 2022
(SIGIR2020) “Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback’’

Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback About This repository accompanies the real-world experiments conducted i

yuta-saito 19 Dec 1, 2022
This is the official pytorch implementation of AutoDebias, an automatic debiasing method for recommendation.

AutoDebias This is the official pytorch implementation of AutoDebias, a debiasing method for recommendation system. AutoDebias is proposed in the pape

Dong Hande 77 Nov 25, 2022
This repository contains the code for "Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP".

Self-Diagnosis and Self-Debiasing This repository contains the source code for Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based

Timo Schick 62 Dec 12, 2022
Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

Deep Deterministic Uncertainty This repository contains the code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic

Jishnu Mukhoti 69 Nov 28, 2022
Study of human inductive biases in CNNs and Transformers.

Are Convolutional Neural Networks or Transformers more like human vision? This repository contains the code and fine-tuned models of popular Convoluti

Shikhar Tuli 39 Dec 8, 2022
How to Become More Salient? Surfacing Representation Biases of the Saliency Prediction Model

How to Become More Salient? Surfacing Representation Biases of the Saliency Prediction Model

Bogdan Kulynych 49 Nov 5, 2022
PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation.

ALiBi PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. Quickstart Clone this reposit

Jake Tae 4 Jul 27, 2022