MiSLAS
Improving Calibration for Long-Tailed Recognition
Authors: Zhisheng Zhong, Jiequan Cui, Shu Liu, Jiaya Jia
Introduction: This repository provides an implementation for the CVPR 2021 paper: "Improving Calibration for Long-Tailed Recognition" based on LDAM-DRW and Decoupling models. Our study shows, because of the extreme imbalanced composition ratio of each class, networks trained on long-tailed datasets are more miscalibrated and over-confident. MiSLAS is a simple, and efficient two-stage framework for long-tailed recognition, which greatly improves recognition accuracy and markedly relieves over-confidence simultaneously.
Installation
Requirements
- Python 3.7
- torchvision 0.4.0
- Pytorch 1.2.0
- yacs 0.1.8
Virtual Environment
conda create -n MiSLAS python==3.7
source activate MiSLAS
Install MiSLAS
git clone https://github.com/Jia-Research-Lab/MiSLAS.git
cd MiSLAS
pip install -r requirements.txt
Dataset Preparation
Change the data_path in config/*/*.yaml accordingly.
Training
Stage-1:
To train a model for Stage-1 with mixup, run:
(one GPU for CIFAR-10-LT & CIFAR-100-LT, four GPUs for ImageNet-LT, iNaturalist 2018, and Places-LT)
python train_stage1.py --cfg ./config/DATASETNAME/DATASETNAME_ARCH_stage1_mixup.yaml
DATASETNAME can be selected from cifar10, cifar100, imagenet, ina2018, and places.
ARCH can be resnet32 for cifar10/100, resnet50/101/152 for imagenet, resnet50 for ina2018, and resnet152 for places, respectively.
Stage-2:
To train a model for Stage-2 with one GPU (all the above datasets), run:
python train_stage2.py --cfg ./config/DATASETNAME/DATASETNAME_ARCH_stage2_mislas.yaml resume /path/to/checkpoint/stage1
The saved folder (including logs and checkpoints) is organized as follows.
MiSLAS
├── saved
│ ├── modelname_date
│ │ ├── ckps
│ │ │ ├── current.pth.tar
│ │ │ └── model_best.pth.tar
│ │ └── logs
│ │ └── modelname.txt
│ ...
Evaluation
To evaluate a trained model, run:
python eval.py --cfg ./config/DATASETNAME/DATASETNAME_ARCH_stage1_mixup.yaml resume /path/to/checkpoint/stage1
python eval.py --cfg ./config/DATASETNAME/DATASETNAME_ARCH_stage2_mislas.yaml resume /path/to/checkpoint/stage2
Results and Models
1) CIFAR-10-LT and CIFAR-100-LT
- Stage-1 (mixup):
| Dataset | Top-1 Accuracy | ECE (15 bins) | Model |
|---|---|---|---|
| CIFAR-10-LT IF=10 | 87.6% | 11.9% | link |
| CIFAR-10-LT IF=50 | 78.1% | 2.49% | link |
| CIFAR-10-LT IF=100 | 72.8% | 2.14% | link |
| CIFAR-100-LT IF=10 | 59.1% | 5.24% | link |
| CIFAR-100-LT IF=50 | 45.4% | 4.33% | link |
| CIFAR-100-LT IF=100 | 39.5% | 8.82% | link |
- Stage-2 (MiSLAS):
| Dataset | Top-1 Accuracy | ECE (15 bins) | Model |
|---|---|---|---|
| CIFAR-10-LT IF=10 | 90.0% | 1.20% | link |
| CIFAR-10-LT IF=50 | 85.7% | 2.01% | link |
| CIFAR-10-LT IF=100 | 82.5% | 3.66% | link |
| CIFAR-100-LT IF=10 | 63.2% | 1.73% | link |
| CIFAR-100-LT IF=50 | 52.3% | 2.47% | link |
| CIFAR-100-LT IF=100 | 47.0% | 4.83% | link |
Note: To obtain better performance, we highly recommend changing the weight decay 2e-4 to 5e-4 on CIFAR-LT.
2) Large-scale Datasets
- Stage-1 (mixup):
| Dataset | Arch | Top-1 Accuracy | ECE (15 bins) | Model |
|---|---|---|---|---|
| ImageNet-LT | ResNet-50 | 45.5% | 7.98% | link |
| iNa'2018 | ResNet-50 | 66.9% | 5.37% | link |
| Places-LT | ResNet-152 | 29.4% | 16.7% | link |
- Stage-2 (MiSLAS):
| Dataset | Arch | Top-1 Accuracy | ECE (15 bins) | Model |
|---|---|---|---|---|
| ImageNet-LT | ResNet-50 | 52.7% | 1.78% | link |
| iNa'2018 | ResNet-50 | 71.6% | 7.67% | link |
| Places-LT | ResNet-152 | 40.4% | 3.41% | link |
Citation
Please consider citing MiSLAS in your publications if it helps your research. :)
@inproceedings{zhong2021mislas,
title={Improving Calibration for Long-Tailed Recognition},
author={Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021},
}
Contact
If you have any questions about our work, feel free to contact us through email (Zhisheng Zhong: [email protected]) or Github issues.
116 Dec 20, 2022
761 Dec 26, 2022
16 Dec 15, 2022
76 Nov 9, 2022
89 Dec 15, 2022
52 Nov 21, 2022
284 Jan 6, 2023
4 Jan 20, 2022
11 Dec 1, 2021
16 Dec 14, 2022
28 Oct 18, 2022
4 Jun 18, 2022
76 Dec 13, 2022
27 Mar 30, 2022
368 Jan 6, 2023
70 Dec 9, 2022
697 Jan 5, 2023
42 Nov 16, 2022
60 Dec 27, 2022