An implementation of the BADGE batch active learning algorithm.

Related tags

Deep Learning badge
Overview

Batch Active learning by Diverse Gradient Embeddings (BADGE)

An implementation of the BADGE batch active learning algorithm. Details are provided in our paper, Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds, which was presented as a talk in ICLR 2020. This code was built by modifying Kuan-Hao Huang's deep active learning repository.

Dependencies

To run this code fully, you'll need PyTorch (we're using version 1.4.0), scikit-learn, and OpenML. We've been running our code in Python 3.7.

Running an experiment

python run.py --model resnet --nQuery 1000 --data CIFAR10 --alg badge
runs an active learning experiment using a ResNet and CIFAR-10 data, querying batches of 1,000 samples according to the BADGE algorithm. This code allows you to also run each of the baseline algorithms used in our paper.

python run.py --model mlp --nQuery 10000 --did 6 --alg conf
runs an active learning experiment using an MLP and dataset number 6 from OpenML, querying batches of 10,000 with confidence sampling. Note that in our code, OpenML datasets can only be used with MLP architectures.

Analyzing experimental results

See the readme file in scripts/ for more details about generating plots like those in our paper.

Comments
  • No pseudo/hypothetical labels founded in the code.

    No pseudo/hypothetical labels founded in the code.

    Hi Jordan Ash,

    In your paper, the gradient embedding is from the loss between the network output and the hypothetical labels (inferred from the network output).

    However, in your code, I didn't find anything about pseudo/hypothetical labels.

    In the file badge_sampling.py, it seems that you directly use the true labels to guide your selection. If so, this would be an unfair comparison.

        gradEmbedding = self.get_grad_embedding(self.X[idxs_unlabeled], self.Y.numpy()[idxs_unlabeled]).numpy()

    I'm not sure if I miss something. Could your show how you use the hypothetical labels in your code?

    Thanks, Rui

    opened by SupeRuier 1
  • Entropy sampling issue

    Entropy sampling issue

    There is an error in your implementation of entropy sampling.

    The following is an example and the result is when using implemented entropy sampling.

    probs = [[0.1, 0.2, 0.7], [0.0, 0.1, 0.9], [0.0, 0.5, 0.5]]
probs = torch.tensor(probs)

log_probs = torch.log(probs)

U = (probs*log_probs).sum(1)

print(log_probs)
print(U)
---------
-- Print Result -- 
tensor([[-2.3026, -1.6094, -0.3567],
        [   -inf, -2.3026, -0.1054],
        [   -inf, -0.6931, -0.6931]])
tensor([-0.8018,     nan,     nan])
------------

    If this issue is not addressed, the results of your implemented entropy sampling will inevitably be much worse than margin sampling.

    This can be solved simply by adding the following code:

    log_probs = torch.log(probs)

log_probs[log_probs == float("-inf")] = 0
log_probs[log_probs == float("inf")] = 0
    opened by ElvinKim 0
  • Does the gradients embedding in the

    Does the gradients embedding in the "get_grad_embedding" are lack of '-' ?

    I try to implement how to get those derivation，but I found I have an additional negative symbol，compared to the author's code. Could anyone account for this problem？Thanks very much.

    opened by czjghost 0
  • Utility of calculating gram matrix in k-means++ method

    Utility of calculating gram matrix in k-means++ method

    Hi Jordan Ash,

    First of all, thank you for sharing such a clear and concise code. It has been a delight while going through the code step by step and understanding it. I was curious about the last steps in kmeans++ code that involves calculating gram matrix followed by eigenvalues calculation. However, I saw no usage of it anywhere while sampling. I wanted to know if its as usual or something is missing in kmeans code.

    Thanks, Shwetank

    opened by pshwetank 0
  • Different results

    Different results

    Hi authors, It's great that you publish the source code, but the default hyper-parameters seem not to be correctly tuned, so could you please share the hyper-params to produce the graph below (taken from your paper) I also add the graph showing the results that I ran with hyper-params from your source code with command python run.py --model vgg --nQuery 100 --data CIFAR10 --alg badge

    Reported in the paper: image Run default code: (the x-axis is the number of data point, divided by 100 - the query batch size) The gap is nearly 10% even with more data image

    opened by hieuvt29 4
  • Large embedding vector, large number of classes?

    Large embedding vector, large number of classes?

    Hi authors, Thanks for pushing this fantastic tool. I have a question regarding to the approach itself: When my my last layer dimension is really large + I have large number of classes to cover, if we compute the gradient vector, its dimension(embDim * nLab) will be of orders of magnitude 10^4 or more. Do you think BADGE is an efficient solution in this case?

    opened by lindsey98 1
Owner
null
GitHub
Custom TensorFlow2 implementations of forward and backward computation of soft-DTW algorithm in batch mode.

Batch Soft-DTW(Dynamic Time Warping) in TensorFlow2 including forward and backward computation Custom TensorFlow2 implementations of forward and backw

null 19 Aug 30, 2022
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

This is the Vowpal Wabbit fast online learning code. Why Vowpal Wabbit? Vowpal Wabbit is a machine learning system which pushes the frontier of machin

Vowpal Wabbit 8.1k Jan 6, 2023
SpecAugmentPyTorch - A Pytorch (support batch and channel) implementation of GoogleBrain's SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

SpecAugment An implementation of SpecAugment for Pytorch How to use Install pytorch, version>=1.9.0 (new feature (torch.Tensor.take_along_dim) is used

IMLHF 3 Oct 11, 2022
[ICLR'21] FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization This is the PyTorch implemention of our paper FedBN: Federated Learning on

Med-AIR@CUHK 156 Dec 15, 2022
Run Effective Large Batch Contrastive Learning on Limited Memory GPU

Gradient Cache Gradient Cache is a simple technique for unlimitedly scaling contrastive learning batch far beyond GPU memory constraint. This means tr

Luyu Gao 198 Dec 29, 2022
PyTorch implementation of "A Simple Baseline for Low-Budget Active Learning".

A Simple Baseline for Low-Budget Active Learning This repository is the implementation of A Simple Baseline for Low-Budget Active Learning. In this pa

null 10 Nov 14, 2022
Tutorial on active learning with the Nvidia Transfer Learning Toolkit (TLT).

Active Learning with the Nvidia TLT Tutorial on active learning with the Nvidia Transfer Learning Toolkit (TLT). In this tutorial, we will show you ho

Lightly 25 Dec 3, 2022
Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready inference.

Yolov5 running on TorchServe (GPU compatible) ! This is a dockerfile to run TorchServe for Yolo v5 object detection model. (TorchServe (PyTorch librar

null 82 Nov 29, 2022
The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Representative Batch Normalization (RBN) with Feature Calibration The official implementation of the CVPR2021 oral paper: Representative Batch Normali

Open source projects of ShangHua-Gao 76 Nov 9, 2022
[CVPRW 21] "BNN - BN = ? Training Binary Neural Networks without Batch Normalization", Tianlong Chen, Zhenyu Zhang, Xu Ouyang, Zechun Liu, Zhiqiang Shen, Zhangyang Wang

BNN - BN = ? Training Binary Neural Networks without Batch Normalization Codes for this paper BNN - BN = ? Training Binary Neural Networks without Bat

VITA 40 Dec 30, 2022
Code used to generate the results appearing in "Train longer, generalize better: closing the generalization gap in large batch training of neural networks"

Train longer, generalize better - Big batch training This is a code repository used to generate the results appearing in "Train longer, generalize bet

Elad Hoffer 145 Sep 16, 2022
Code for "On the Effects of Batch and Weight Normalization in Generative Adversarial Networks"

Note: this repo has been discontinued, please check code for newer version of the paper here Weight Normalized GAN Code for the paper "On the Effects

Sitao Xiang 182 Sep 6, 2021
Training vision models with full-batch gradient descent and regularization

Stochastic Training is Not Necessary for Generalization -- Training competitive vision models without stochasticity This repository implements trainin

Jonas Geiping 32 Jan 6, 2023
Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

null 37 Dec 3, 2022
RL algorithm PPO and IRL algorithm AIRL written with Tensorflow.

RL algorithm PPO and IRL algorithm AIRL written with Tensorflow. They have a parallel sampling feature in order to increase computation speed (especially in high-performance computing (HPC)).

Fangjian Li 3 Dec 28, 2021
A modular active learning framework for Python

Modular Active Learning framework for Python3 Page contents Introduction Active learning from bird's-eye view modAL in action From zero to one in a fe

modAL 1.9k Dec 31, 2022
Code for Multiple Instance Active Learning for Object Detection, CVPR 2021

Language: 简体中文 | English Introduction This is the code for Multiple Instance Active Learning for Object Detection, CVPR 2021. Installation A Linux pla

Tianning Yuan 269 Dec 21, 2022
Code for Multiple Instance Active Learning for Object Detection, CVPR 2021

MI-AOD Language: 简体中文 | English Introduction This is the code for Multiple Instance Active Learning for Object Detection (The PDF is not available tem

Tianning Yuan 269 Dec 21, 2022
Official PyTorch code for CVPR 2020 paper "Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision"

Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision https://arxiv.org/abs/2003.00393 Abstract Active learning (AL) aims to min

Denis 29 Nov 21, 2022