LiST (Lite Self-Training)
This is the implementation of the paper LiST: Lite Self-training Makes Efficient Few-shot Learners. LiST is short for Lite Self-Training.
Overview
Setup Environment
Install via pip:
- create a conda environment running Python 3.7:
conda create --name LiST python=3.7
conda activate LiST
- install the required dependencies:
pip install -r requirements.txt
Use docker:
- Pull docker
docker pull yaqing/pytorch-few-shot:v0.6
- Run docker
docker run -it --rm --runtime nvidia yaqing/pytorch-few-shot:v0.6 bash
Please refer to the following link if you first use docker: https://docs.docker.com/
NOTE: Different versions of packages (like pytorch, transformers, etc.) may lead to different results from the paper. However, the trend should still hold no matter what versions of packages you use.
Prepare the data
Please run the following commands to prepare data for experiments:
cd data
bash prepare_dataset.sh
cd ..
Run the model
We prepare scripts to run tasks. Please use bash script under LiST directory.
Run LiST as:
bash run.sh
Note that we ran experiments on V100 GPU (32GB) for LiST models. You may need to reduce batch size for other GPUs.
Supported datasets:
MNLI, RTE, QQP, SST-2, subj and MPQA with shots of 10, 20, 30.
Notes and Acknowledgments
The implementation is based on https://github.com/huggingface/transformers
We also used some code from: https://github.com/princeton-nlp/LM-BFF
How do I cite LiST?
@article{wang2021list,
title={LiST: Lite Self-training Makes Efficient Few-shot Learners},
author={Wang, Yaqing and Mukherjee, Subhabrata and Liu, Xiaodong and Gao, Jing and Awadallah, Ahmed Hassan and Gao, Jianfeng},
journal={arXiv preprint arXiv:2110.06274},
year={2021}
}
80 Dec 27, 2022
33 Oct 31, 2022
78 Dec 2, 2022
73 Dec 18, 2022
36 Oct 30, 2022
2.4k Jan 5, 2023
319 Dec 30, 2022
9 Nov 7, 2022
8.4k Jan 1, 2023
34 Oct 8, 2022
8 Oct 14, 2021
246 Dec 19, 2022
238 Jan 4, 2023
8 Dec 11, 2022
103 Jan 5, 2023
2.3k Jan 4, 2023
214 Dec 29, 2022
138 Dec 12, 2022