TextFooler
A Model for Natural Language Attack on Text Classification and Inference
This is the source code for the paper: Jin, Di, et al. "Is BERT Really Robust? Natural Language Attack on Text Classification and Entailment." arXiv preprint arXiv:1907.11932 (2019). If you use the code, please cite the paper:
@article{jin2019bert,
title={Is BERT Really Robust? Natural Language Attack on Text Classification and Entailment},
author={Jin, Di and Jin, Zhijing and Zhou, Joey Tianyi and Szolovits, Peter},
journal={arXiv preprint arXiv:1907.11932},
year={2019}
}
Data
Our 7 datasets are here.
Prerequisites:
Required packages are listed in the requirements.txt file:
pip install -r requirements.txt
How to use
- Run the following code to install the esim package:
cd ESIM
python setup.py install
cd ..
- (Optional) Run the following code to pre-compute the cosine similarity scores between word pairs based on the counter-fitting word embeddings.
python comp_cos_sim_mat.py [PATH_TO_COUNTER_FITTING_WORD_EMBEDDINGS]
- Run the following code to generate the adversaries for text classification:
python attack_classification.py
For Natural langauge inference:
python attack_nli.py
Examples of run code for these two files are in run_attack_classification.py and run_attack_nli.py. Here we explain each required argument in details:
- --dataset_path: The path to the dataset. We put the 1000 examples for each dataset we used in the paper in the folder data.
- --target_model: Name of the target model such as ''bert''.
- --target_model_path: The path to the trained parameters of the target model. For ease of replication, we shared the trained BERT model parameters, the trained LSTM model parameters, and the trained CNN model parameters on each dataset we used.
- --counter_fitting_embeddings_path: The path to the counter-fitting word embeddings.
- --counter_fitting_cos_sim_path: This is optional. If given, then the pre-computed cosine similarity scores based on the counter-fitting word embeddings will be loaded to save time. If not, it will be calculated.
- --USE_cache_path: The path to save the USE model file (Downloading is automatic if this path is empty).
Two more things to share with you:
-
In case someone wants to replicate our experiments for training the target models, we shared the used seven datasets we have processed for you!
-
In case someone may want to use our generated adversary results towards the benchmark data directly, here it is.
82 Nov 29, 2022
14 Sep 1, 2022
92 Jan 7, 2023
157 Dec 11, 2022
8 Oct 3, 2022
3 Feb 8, 2022
119 Dec 2, 2022
52 Jan 7, 2023
35 Oct 31, 2022
18 Nov 21, 2022
6 Aug 1, 2022
109 Dec 28, 2022
105 Jan 3, 2023
7 Aug 16, 2022
10 Nov 14, 2022
11 Mar 20, 2022
0 Jan 23, 2022
81 Dec 12, 2022
4 Jun 28, 2022