Classifier-Balancing
This repository contains code for the paper:
Decoupling Representation and Classifier for Long-Tailed Recognition
Bingyi Kang, Saining Xie,Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, Yannis Kalantidis
[OpenReview] [Arxiv] [PDF] [Slides] [@ICLR]
Facebook AI Research, National University of Singapore
International Conference on Learning Representations (ICLR), 2020
Abstract
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but all of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability with relative ease by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.
If you find this code useful, consider citing our work:
@inproceedings{kang2019decoupling,
title={Decoupling representation and classifier for long-tailed recognition},
author={Kang, Bingyi and Xie, Saining and Rohrbach, Marcus and Yan, Zhicheng
and Gordo, Albert and Feng, Jiashi and Kalantidis, Yannis},
booktitle={Eighth International Conference on Learning Representations (ICLR)},
year={2020}
}
Requirements
The code is based on https://github.com/zhmiao/OpenLongTailRecognition-OLTR.
Dataset
-
ImageNet_LT and Places_LT
Download the ImageNet_2014 and Places_365.
-
iNaturalist 2018
Change the data_root in main.py accordingly.
Representation Learning
- Instance-balanced Sampling
python main.py --cfg ./config/ImageNet_LT/feat_uniform.yaml
- Class-balanced Sampling
python main.py --cfg ./config/ImageNet_LT/feat_balance.yaml
- Square-root Sampling
python main.py --cfg ./config/ImageNet_LT/feat_squareroot.yaml
- Progressively-balancing Sampling
python main.py --cfg ./config/ImageNet_LT/feat_shift.yaml
Test the joint learned classifier with representation learning
python main.py --cfg ./config/ImageNet_LT/feat_uniform.yaml --test
Classifier Learning
- Nearest Class Mean classifier (NCM).
python main.py --cfg ./config/ImageNet_LT/feat_uniform.yaml --test --knn
- Classifier Re-training (cRT)
python main.py --cfg ./config/ImageNet_LT/cls_crt.yaml --model_dir ./logs/ImageNet_LT/models/resnext50_uniform_e90
python main.py --cfg ./config/ImageNet_LT/cls_crt.yaml --test
- Tau-normalization
Extract fatures
for split in train_split val test
do
python main.py --cfg ./config/ImageNet_LT/feat_uniform.yaml --test --save_feat $split
done
Evaluation
for split in train val test
do
python tau_norm.py --root ./logs/ImageNet_LT/models/resnext50_uniform_e90/ --type $split
done
- Learnable weight scaling (LWS)
python main.py --cfg ./config/ImageNet_LT/cls_lws.yaml --model_dir ./logs/ImageNet_LT/models/resnext50_uniform_e90
python main.py --cfg ./config/ImageNet_LT/cls_lws.yaml --test
Results and Models
ImageNet_LT
-
Representation learning
Sampling Many Medium Few All Model Instance-Balanced 65.9 37.5 7.7 44.4 ResNeXt50 Class-Balanced 61.8 40.1 15.5 45.1 ResNeXt50 Square-Root 64.3 41.2 17.0 46.8 ResNeXt50 Progressively-Balanced 61.9 43.2 19.4 47.2 ResNeXt50 For other models trained with instance-balanced (natural) sampling:
[ResNet50] [ResNet101] [ResNet152] [ResNeXt101] [ResNeXt152] -
Classifier learning
Classifier Many Medium Few All Model Joint 65.9 37.5 7.7 44.4 ResNeXt50 NCM 56.6 45.3 28.1 47.3 ResNeXt50 cRT 61.8 46.2 27.4 49.6 ResNeXt50 Tau-normalization 59.1 46.9 30.7 49.4 ResNeXt50 LWS 60.2 47.2 30.3 49.9 ResNeXt50
iNaturalist 2018
- Representaion Learning
We provide the pre-trained models with instance-balanced (natural) sampling for 90 and 200 epochs as follows:
90 epochs: [ResNet50] [ResNet101] [ResNet152] 200 epochs: [ResNet50] [ResNet101] [ResNet152] - Classifier learning
We provide the cRT and LWS models based on a pretrained ResNet152 model (200 epochs) as follows:
[ResNet152(cRT)] [ResNet152(LWS)]
Places_LT
- Representaion learning
We provide a pretrained ResNet152 with instance-balanced (natural) sampling: [link] - Classifier learning
We provide the cRT and LWS models based on above pretrained ResNet152 model as follows:
[ResNet152(cRT)] [ResNet152(LWS)]
To test a pretrained model:
python main.py --cfg /path/to/config/file --model_dir /path/to/model/file --test
License
This project is licensed under the license found in the LICENSE file in the root directory of this source tree (here). Portions of the source code are from the OLTR project.
88 Nov 22, 2022
697 Jan 6, 2023
36 Oct 19, 2022
69 Nov 26, 2022
20 Sep 8, 2022
32 Nov 29, 2022
70 Nov 16, 2022
31 Jan 6, 2023
34 Nov 16, 2022
148 Dec 21, 2022
11 Nov 19, 2022
33 Dec 18, 2022
40 Dec 5, 2022
197 Nov 29, 2022
728 Dec 28, 2022
281 Dec 22, 2022
65 Dec 27, 2022
89 Dec 26, 2022