Can Active Learning Preemptively Mitigate Fairness Issues?
Code for the paper "Can Active Learning Preemptively Mitigate Fairness Issues?" presented at RAI 2021.
Arxiv link: https://arxiv.org/pdf/2104.06879.pdf
This repo aims at helping researchers reproduce our paper. We welcome questions and suggestions, please submit an issue! If you are working in active learning, our library BaaL might help you!
Glossary
- query_size : Number of data labelled per AL step
- AL Step: process of training, selecting samples, and labelling.
- learning_epoch: Number of epoch to train the model before uncertainty estimation.
Datasets
We experimented with many datasets and we have one format.
# For synbols datasets
with open(dataset_path, 'rb') as f:
train_ds = pickle.load(f)
test_ds = pickle.load(f)
val_ds = pickle.load(f)
x,y = train_ds
print(x[0])
# A numpy array with shape [64,64,3]
print(y[0])
# A dictionnary with all the attributes and keys.
"""
{'char':'a',
'color': 'r',
...
}
"""
Models
We use a VGG-16 and we do AL with MC-Dropout.
from active_fairness import utils
from baal.bayesian.dropout import patch_module
hyperparams = {} # Dictionnary for the hparams
model = utils.vgg16(pretrained=hyperparams['pretrained'],
num_classes=hyperparams['n_cls'])
# change dropout layer to be able to use MC-Dropout
model = patch_module(model)
How to launch
Here is an example to run our experiment on a particular dataset.
To generate the datasets, please look in misc/biased_dataset.ipynb.
poetry run python experiments/grad_experiment.py datasets/minority_dataset_50000.pkl color char
8 Aug 2, 2022
210 Dec 30, 2022
199 Jan 8, 2023
366 Dec 28, 2022
59 Oct 31, 2022
22 Sep 8, 2022
334 Dec 23, 2022
174 Dec 29, 2022
55 Nov 14, 2022
11 Oct 23, 2022
5 Nov 30, 2022
29 Sep 2, 2022
76 Jan 3, 2023
22 Nov 16, 2022
3 Dec 2, 2022
41 Apr 28, 2022
3 Jan 3, 2023
18 Oct 6, 2022
66 Dec 26, 2022