Spinning Language Models for Propaganda-As-A-Service
This is the source code for the Arxiv version of the paper. You can use this Google Colab to explore the results. Spinned models are located on HuggingFace Hub.
Please feel free to contact me: [email protected].
Ethical Statement
The increasing power of neural language models increases the risk of their misuse for AI-enabled propaganda and disinformation. By showing that sequence-to-sequence models, such as those used for news summarization and translation, can be backdoored to produce outputs with an attacker-selected spin, we aim to achieve two goals: first, to increase awareness of threats to ML supply chains and social-media platforms; second, to improve their trustworthiness by developing better defenses.
Repo details
This repo is a fork from Huggingface transformers at version 4.11.0.dev0 commit. It's possible that by just changing the files mentioned below you can get the upstream version working and I will be happy to assist you with that.
Details to spin your own models.
Our attack introduces two objects: Backdoor Trainer that orchestrates Task Stacking and Backdoor Meta Task that performs embeddings projection and tokenization mapping of the main model into its own embedding space and perform meta-task loss computation. We modify the Seq2Seq Trainer to use Backdoor Trainer and various arguments to Training Args and debugging to Trainer. Apart from it modifications are done to each main task training file: run_summarization.py, run_translation.py, and run_clm.py such that we correctly create datasets and measure performance.
To install create new environment and install package:
conda create -n myenv python=3.8
pip install datasets==1.14.0 names_dataset torch absl-py tensorflow git pyarrow==5.0.0
pip install -e .
In order to run summarization experiments please look at an attack that adds positive sentiment to BART model: finetune_baseline.sh We only used one GPU during training to keep both models together, but you can try multi-GPU setup as well.
cd examples/pytorch/summarization/
pip install -r requirements.txt
mkdir saved_models
CUDA_VISIBLE_DEVICES=0 sh finetune_baseline.sh
Similarly, you can run Toxicity at finetune_toxic.sh and Entailment at finetune_mnli.sh
For translation you need to use finetune_translate.sh
cd examples/pytorch/translation/
pip install -r requirements.txt
mkdir saved_models
CUDA_VISIBLE_DEVICES=0 sh finetune_translate.sh
And language experiments with GPT-2 can be run using finetune_clm.sh:
cd examples/pytorch/language-modeling/
pip install -r requirements.txt
mkdir saved_models
CUDA_VISIBLE_DEVICES=0 sh finetune_clm.sh
Citation
@article{bagdasaryan2021spinning,
title={Spinning Sequence-to-Sequence Models with Meta-Backdoors},
author={Bagdasaryan, Eugene and Shmatikov, Vitaly},
journal={arXiv preprint arXiv:2112.05224},
year={2021}
}
43 Nov 19, 2022
37 Nov 21, 2022
199 Jan 8, 2023
296 Dec 29, 2022
138 Apr 15, 2022
56 Nov 28, 2022
59 Oct 31, 2022
726 Jan 5, 2023
66 Dec 26, 2022
29 Dec 28, 2022
73 Nov 6, 2022
1 Oct 26, 2021
32 Nov 9, 2021
138 Dec 12, 2022
64 Nov 11, 2022
203 Jan 8, 2023
210 Dec 30, 2022
139 Nov 18, 2022