Pytorch implementation of TailCalibX : Feature Generation for Long-tail Classification

Overview

TailCalibX : Feature Generation for Long-tail Classification

by Rahul Vigneswaran, Marc T. Law, Vineeth N. Balasubramanian, Makarand Tapaswi

[arXiv] [Code] [pip Package] [Video] TailCalibX methodology

Table of contents

๐Ÿฃ Easy Usage (Recommended way to use our method)

โš  Caution: TailCalibX is just TailCalib employed multiple times. Specifically, we generate a set of features once every epoch and use them to train the classifier. In order to mimic that, three things must be done at every epoch in the following order:

  1. Collect all the features from your dataloader.
  2. Use the tailcalib package to make the features balanced by generating samples.
  3. Train the classifier.
  4. Repeat.

๐Ÿ’ป Installation

Use the package manager pip to install tailcalib.

pip install tailcalib

๐Ÿ‘จโ€๐Ÿ’ป Example Code

Check the instruction here for a much more detailed python package information.

# Import
from tailcalib import tailcalib

# Initialize
a = tailcalib(base_engine="numpy")   # Options: "numpy", "pytorch"

# Imbalanced random fake data
import numpy as np
X = np.random.rand(200,100)
y = np.random.randint(0,10, (200,))

# Balancing the data using "tailcalib"
feat, lab, gen = a.generate(X=X, y=y)

# Output comparison
print(f"Before: {np.unique(y, return_counts=True)}")
print(f"After: {np.unique(lab, return_counts=True)}")

๐Ÿงช Advanced Usage

โœ” Things to do before you run the code from this repo

  • Change the data_root for your dataset in main.py.
  • If you are using wandb logging (Weights & Biases), make sure to change the wandb.init in main.py accordingly.

๐Ÿ“€ How to use?

  • For just the methods proposed in this paper :
    • For CIFAR100-LT: run_TailCalibX_CIFAR100-LT.sh
    • For mini-ImageNet-LT : run_TailCalibX_mini-ImageNet-LT.sh
  • For all the results show in the paper :
    • For CIFAR100-LT: run_all_CIFAR100-LT.sh
    • For mini-ImageNet-LT : run_all_mini-ImageNet-LT.sh

๐Ÿ“š How to create the mini-ImageNet-LT dataset?

Check Notebooks/Create_mini-ImageNet-LT.ipynb for the script that generates the mini-ImageNet-LT dataset with varying imbalance ratios and train-test-val splits.

โš™ Arguments

  • --seed : Select seed for fixing it.

    • Default : 1
  • --gpu : Select the GPUs to be used.

    • Default : "0,1,2,3"
  • --experiment: Experiment number (Check 'libs/utils/experiment_maker.py').

    • Default : 0.1
  • --dataset : Dataset number.

    • Choices : 0 - CIFAR100, 1 - mini-imagenet
    • Default : 0
  • --imbalance : Select Imbalance factor.

    • Choices : 0: 1, 1: 100, 2: 50, 3: 10
    • Default : 1
  • --type_of_val : Choose which dataset split to use.

    • Choices: "vt": val_from_test, "vtr": val_from_train, "vit": val_is_test
    • Default : "vit"
  • --cv1 to --cv9 : Custom variable to use in experiments - purpose changes according to the experiment.

    • Default : "1"
  • --train : Run training sequence

    • Default : False
  • --generate : Run generation sequence

    • Default : False
  • --retraining : Run retraining sequence

    • Default : False
  • --resume : Will resume from the 'latest_model_checkpoint.pth' and wandb if applicable.

    • Default : False
  • --save_features : Collect feature representations.

    • Default : False
  • --save_features_phase : Dataset split of representations to collect.

    • Choices : "train", "val", "test"
    • Default : "train"
  • --config : If you have a yaml file with appropriate config, provide the path here. Will override the 'experiment_maker'.

    • Default : None

๐Ÿ‹๏ธโ€โ™‚๏ธ Trained weights

Experiment CIFAR100-LT (ResNet32, seed 1, Imb 100) mini-ImageNet-LT (ResNeXt50)
TailCalib Git-LFS Git-LFS
TailCalibX Git-LFS Git-LFS
CBD + TailCalibX Git-LFS Git-LFS

๐Ÿช€ Results on a Toy Dataset

Open In Colab

The higher the Imb ratio, the more imbalanced the dataset is. Imb ratio = maximum_sample_count / minimum_sample_count.

Check this notebook to play with the toy example from which the plot below was generated.

๐ŸŒด Directory Tree

TailCalibX
โ”œโ”€โ”€ libs
โ”‚   โ”œโ”€โ”€ core
โ”‚   โ”‚   โ”œโ”€โ”€ ce.py
โ”‚   โ”‚   โ”œโ”€โ”€ core_base.py
โ”‚   โ”‚   โ”œโ”€โ”€ ecbd.py
โ”‚   โ”‚   โ”œโ”€โ”€ modals.py
โ”‚   โ”‚   โ”œโ”€โ”€ TailCalib.py
โ”‚   โ”‚   โ””โ”€โ”€ TailCalibX.py
โ”‚   โ”œโ”€โ”€ data
โ”‚   โ”‚   โ”œโ”€โ”€ dataloader.py
โ”‚   โ”‚   โ”œโ”€โ”€ ImbalanceCIFAR.py
โ”‚   โ”‚   โ””โ”€โ”€ mini-imagenet
โ”‚   โ”‚       โ”œโ”€โ”€ 0.01_test.txt
โ”‚   โ”‚       โ”œโ”€โ”€ 0.01_train.txt
โ”‚   โ”‚       โ””โ”€โ”€ 0.01_val.txt
โ”‚   โ”œโ”€โ”€ loss
โ”‚   โ”‚   โ”œโ”€โ”€ CosineDistill.py
โ”‚   โ”‚   โ””โ”€โ”€ SoftmaxLoss.py
โ”‚   โ”œโ”€โ”€ models
โ”‚   โ”‚   โ”œโ”€โ”€ CosineDotProductClassifier.py
โ”‚   โ”‚   โ”œโ”€โ”€ DotProductClassifier.py
โ”‚   โ”‚   โ”œโ”€โ”€ ecbd_converter.py
โ”‚   โ”‚   โ”œโ”€โ”€ ResNet32Feature.py
โ”‚   โ”‚   โ”œโ”€โ”€ ResNext50Feature.py
โ”‚   โ”‚   โ””โ”€โ”€ ResNextFeature.py
โ”‚   โ”œโ”€โ”€ samplers
โ”‚   โ”‚   โ””โ”€โ”€ ClassAwareSampler.py
โ”‚   โ””โ”€โ”€ utils
โ”‚       โ”œโ”€โ”€ Default_config.yaml
โ”‚       โ”œโ”€โ”€ experiments_maker.py
โ”‚       โ”œโ”€โ”€ globals.py
โ”‚       โ”œโ”€โ”€ logger.py
โ”‚       โ””โ”€โ”€ utils.py
โ”œโ”€โ”€ LICENSE
โ”œโ”€โ”€ main.py
โ”œโ”€โ”€ Notebooks
โ”‚   โ”œโ”€โ”€ Create_mini-ImageNet-LT.ipynb
โ”‚   โ””โ”€โ”€ toy_example.ipynb
โ”œโ”€โ”€ readme_assets
โ”‚   โ”œโ”€โ”€ method.svg
โ”‚   โ””โ”€โ”€ toy_example_output.svg
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ run_all_CIFAR100-LT.sh
โ”œโ”€โ”€ run_all_mini-ImageNet-LT.sh
โ”œโ”€โ”€ run_TailCalibX_CIFAR100-LT.sh
โ””โ”€โ”€ run_TailCalibX_mini-imagenet-LT.sh

Ignored tailcalib_pip as it is for the tailcalib pip package.

๐Ÿ“ƒ Citation

@inproceedings{rahul2021tailcalibX,
    title   = {{Feature Generation for Long-tail Classification}},
    author  = {Rahul Vigneswaran and Marc T. Law and Vineeth N. Balasubramanian and Makarand Tapaswi},
    booktitle = {ICVGIP},
    year = {2021}
}

๐Ÿ‘ Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

โค About me

Rahul Vigneswaran

โœจ Extras

๐Ÿ Long-tail buzz : If you are interested in deep learning research which involves long-tailed / imbalanced dataset, take a look at Long-tail buzz to learn about the recent trending papers in this field.

๐Ÿ“ License

MIT

You might also like...
Official implementation of
Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation" (SIGGRAPH 2021)

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation This repository contains the official PyTorch implementation of the following

Official repository for "PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation"

pair-emnlp2020 Official repository for the paper: Xinyu Hua and Lu Wang: PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long

Simple-Image-Classification - Simple Image Classification Code (PyTorch)
Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Simple-Image-Classification Simple Image Classification Code (PyTorch) Yechan Kim This repository contains: Python3 / Pytorch code for multi-class ima

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''
A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

README.md shall be finished soon. WSSGG 0 Overview 1 Installation 1.1 Faster-RCNN 1.2 Language Parser 1.3 GloVe Embeddings 2 Settings 2.1 VG-GT-Graph

Working demo of the Multi-class and Anomaly classification model using the CLIP feature space
Working demo of the Multi-class and Anomaly classification model using the CLIP feature space

๐Ÿ‘๏ธ Hindsight AI: Crime Classification With Clip About For Educational Purposes Only This is a recursive neural net trained to classify specific crime

Pytorch implementation for
Pytorch implementation for "Adversarial Robustness under Long-Tailed Distribution" (CVPR 2021 Oral)

Adversarial Long-Tail This repository contains the PyTorch implementation of the paper: Adversarial Robustness under Long-Tailed Distribution, CVPR 20

Official implementation of Long-Short Transformer in PyTorch.
Official implementation of Long-Short Transformer in PyTorch.

Long-Short Transformer (Transformer-LS) This repository hosts the code and models for the paper: Long-Short Transformer: Efficient Transformers for La

Pytorch implementation for
Pytorch implementation for "Large-Scale Long-Tailed Recognition in an Open World" (CVPR 2019 ORAL)

Large-Scale Long-Tailed Recognition in an Open World [Project] [Paper] [Blog] Overview Open Long-Tailed Recognition (OLTR) is the author's re-implemen

PyTorch implementation for our NeurIPS 2021 Spotlight paper
PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short-Term Transformer for Online Action Detection".

Long Short-Term Transformer for Online Action Detection Introduction This is a PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short

Comments
  • File Not found error

    File Not found error

    Hi, while executing the code, it generates the following error. The file is also present in the current directory. I was just wondering if anyone could help with this.

    image

    bug 
    opened by Angelina1996 2
  • Creatin mini ImageNet_LT

    Creatin mini ImageNet_LT

    Hi Rahul, Just wondering if you could help with the following error in creating a mini imageNet_LT dataset. I am using imageNet2012 which has 2 directories, imageNet train and val. ImageNet Directory pic is also provided below the error message. Error (base) [Angelina@rob-gpu Notebooks]$ python create_mini_ImageNet_LT.py Traceback (most recent call last): File "create_mini_ImageNet_LT.py", line 69, in train_x, test_x, train_y, test_y = train_test_split(final_1, labels_1, train_size=train_split_ratio, test_size=test_split_ratio, stratify=labels_1) File "/home/Angelina/anaconda3/lib/python3.8/site-packages/sklearn/model_selection/_split.py", line 2130, in train_test_split n_train, n_test = _validate_shuffle_split(n_samples, test_size, train_size, File "/home/Angelina/anaconda3/lib/python3.8/site-packages/sklearn/model_selection/_split.py", line 1810, in _validate_shuffle_split raise ValueError( ValueError: With n_samples=0, test_size=0.19999999999999996 and train_size=0.8, the resulting train set will be empty. Adjust any of the aforementioned parameters.

    opened by Angelina1996 2
  • The experiments of CE+TailCalib

    The experiments of CE+TailCalib

    Thanks for your excellent work! But I have a question about experiments of CE+TailCalib. I want to find this experiment, but it seems that not including in below. So how can I find the information about this experiment? Thank you! image

    reproducibility 
    opened by madoka109 1
  • question about cosineCE

    question about cosineCE

    I have run the CIFAR-100-Imb100 experiment_no=0.2, i got this results which is different from that in the paper. CIFAR100 | Imb100 | val-toatal | val-many | val-med | val-few -- | -- | -- | -- | -- | -- Seed1 | CosineCE | 0.403 | 0.676 | 0.379 | 0.112 Seed2 | CosineCE | 0.403 | 0.670 | 0.396 | 0.101 Seed3 | CosineCE | 0.405 | 0.680 | 0.380 | 0.113 Seed10 | CosineCE | 0.4051 | 0.6737 | 0.3923 | 0.1067 Seed20 | CosineCE | 0.3968 | 0.6754 | 0.3909 | 0.7867 Seed30 | CosineCE | 0.4045 | 0.6811 | 0.378 | 0.1127

    reproducibility 
    opened by Z-ZHHH 3
Owner
Rahul Vigneswaran
Rahul Vigneswaran
PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment

logit-adj-pytorch PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment This code implements the paper: Long-tail Learning via

Chamuditha Jayanga 53 Dec 23, 2022
Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

null 105 Nov 7, 2022
Official implementation for CVPR 2021 paper: Adaptive Class Suppression Loss for Long-Tail Object Detection

Adaptive Class Suppression Loss for Long-Tail Object Detection This repo is the official implementation for CVPR 2021 paper: Adaptive Class Suppressio

CASIA-IVA-Lab 67 Dec 4, 2022
A scientific and useful toolbox, which contains practical and effective long-tail related tricks with extensive experimental results

Bag of tricks for long-tailed visual recognition with deep convolutional neural networks This repository is the official PyTorch implementation of AAA

Yong-Shun Zhang 181 Dec 28, 2022
A coin flip game in which you can put the amount of money below or equal to 1000 and then choose heads or tail

COIN_FLIPPY ##This is a simple example package. You can use Github-flavored Markdown to write your content. Coinflippy A coin flip game in which you c

null 2 Dec 26, 2021
Pytorch implementation of the AAAI 2022 paper "Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification"

[AAAI22] Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification We point out the overlooked unbiasedness in long-tailed clas

PatatiPatata 28 Oct 18, 2022
Exploring Classification Equilibrium in Long-Tailed Object Detection, ICCV2021

Exploring Classification Equilibrium in Long-Tailed Object Detection (LOCE, ICCV 2021) Paper Introduction The conventional detectors tend to make imba

null 52 Nov 21, 2022
Code for the AAAI-2022 paper: Imagine by Reasoning: A Reasoning-Based Implicit Semantic Data Augmentation for Long-Tailed Classification

Imagine by Reasoning: A Reasoning-Based Implicit Semantic Data Augmentation for Long-Tailed Classification (AAAI 2022) Prerequisite PyTorch >= 1.2.0 P

null 16 Dec 14, 2022
On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks We provide the code (in PyTorch) and datasets for our paper "On Size-Orient

Zemin Liu 4 Jun 18, 2022
Official implementation of "StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation" (SIGGRAPH 2021)

StyleCariGAN in PyTorch Official implementation of StyleCariGAN:Caricature Generation via StyleGAN Feature Map Modulation in PyTorch Requirements PyTo

PeterZhouSZ 49 Oct 31, 2022