Class-Balanced Loss Based on Effective Number of Samples. CVPR 2019

Last update: Jan 8, 2023

Overview

Class-Balanced Loss Based on Effective Number of Samples

Tensorflow code for the paper:

Class-Balanced Loss Based on Effective Number of Samples
Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, Serge Belongie

Dependencies:

Python (3.6)
Tensorflow (1.14)

Datasets:

Long-Tailed CIFAR. We provide a download link that includes all the data used in our paper in .tfrecords format. The data was converted and generated by src/generate_cifar_tfrecords.py (original CIFAR) and src/generate_cifar_tfrecords_im.py (long-tailed CIFAR).

Effective Number of Samples:

For a visualization of the data and effective number of samples, please take a look at data.ipynb.

Key Implementation Details:

Training and Evaluation:

We provide 3 .sh scripts for training and evaluation.

On original CIFAR dataset:

./cifar_trainval.sh

On long-tailed CIFAR dataset (the hyperparameter IM_FACTOR is the inverse of "Imbalance Factor" in the paper):

./cifar_im_trainval.sh

On long-tailed CIFAR dataset using the proposed class-balanced loss (set non-zero BETA):

./cifar_im_trainval_cb.sh

Run Tensorboard for visualization:

tensorboard --logdir=./results --port=6006

The figure below are the results of running ./cifar_im_trainval.sh and ./cifar_im_trainval_cb.sh:

Training with TPU:

We train networks on iNaturalist and ImageNet datasets using Google's Cloud TPU. The code for this section is in tpu/. Our code is based on the official implementation of Training ResNet on Cloud TPU and forked from https://github.com/tensorflow/tpu.

Data Preparation:

Download datasets (except images) from this link and unzip it under tpu/. The unzipped directory tpu/raw_data/ contains the training and validation splits. For raw images, please download from the following links and put them into the corresponding folders in tpu/raw_data/:
Convert datasets into .tfrecords format and upload to Google Cloud Storage (gcs) using tpu/tools/datasets/dataset_to_gcs.py:

python dataset_to_gcs.py \
  --project=$PROJECT \
  --gcs_output_path=$GCS_DATA_DIR \
  --local_scratch_dir=$LOCAL_TFRECORD_DIR \
  --raw_data_dir=$LOCAL_RAWDATA_DIR

The following 3 .sh scripts in tpu/ can be used to train and evaluate models on iNaturalist and ImageNet using Cloud TPU. For more details on how to use Cloud TPU, please refer to Training ResNet on Cloud TPU.

Note that the image mean and standard deviation and input size need to be updated accordingly.

On ImageNet (ILSVRC 2012):

./run_ILSVRC2012.sh

On iNaturalist 2017:

./run_inat2017.sh

On iNaturalist 2018:

./run_inat2018.sh

The pre-trained models, including all logs viewable on tensorboard, can be downloaded from the following links:

Dataset	Network	Loss	Input Size	Download Link
ILSVRC 2012	ResNet-50	Class-Balanced Focal Loss	224	link
iNaturalist 2018	ResNet-50	Class-Balanced Focal Loss	224	link

Citation

If you find our work helpful in your research, please cite it as:

@inproceedings{cui2019classbalancedloss,
  title={Class-Balanced Loss Based on Effective Number of Samples},
  author={Cui, Yin and Jia, Menglin and Lin, Tsung-Yi and Song, Yang and Belongie, Serge},
  booktitle={CVPR},
  year={2019}
}

Comments

baseline for long-tail cifar10 is 77.47%

Because my implementation for cb-focal in pytorch can only reach 77% accuracy for long-tail cifar10. I have ran your source code without any change. But the baseline is 77.5% (My baseline for pytorch in actually 75%.), which is almost 2.7% higher than the results reported in your paper. Does this mean cb-focal is only 2% higher than baseline?

opened by hbsz123 11
Question About balanced Loss

Hi,

thanks for sharing the code and for your great work. I have a question about the loss in your paper why you add +1 in (1) formula exactly (1-p)(En-1+1). Where does it came from? is the initial expected value?

Thanks

opened by eric-tc 2
CB-Loss makes no sense when n_y is large

When n_y is large, it seems that the loss weights α are always equal to 1. If so, CB Loss makes no sense. @richardaecn For example, 10,000 images belong to class A, and only 1,000 images belong to class B. Then the CB weights are [1,1], no matter how much β is. Please correct me if I have any misunderstanding. Thanks.

opened by eugenelawrence 2
About data augmentation

Hi, may I ask a question about the description in the paper?

As you mentioned in paper in the last paragraph of chapter 3.1:

"the stronger the data augmentation is, the smaller the N will be. The small neighboring region of a sample is a way to capture all near-duplicates and instances that can be obtained by data augmentation"

Shouldn't stronger data augmentation technique provide more samples of the same class(S) that makes the volume of them (N) larger?

opened by guantinglin 1
I'm sorry to disturb you here

I found you might be the organizer of the COCO Detection Challenge. The codalab has closed the old website and your competition can‘t submit.

Can you transfer the competition to the new website? Refer the issue here.

opened by alpc111 0
Performance issues in tpu/models/ (by P3)
Hello! I've found a performance issue in tpu/models/: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.

Detailed description is listed below:

official/retinanet/dataloader.py/: dataset.batch(batch_size, drop_remainder=True)(here) should be called before dataset.map(_dataset_parser, num_parallel_calls=64)(here).

official/retinanet/dataloader.py/: dataset.batch(batch_size, drop_remainder=True)(here) should be called before dataset.map(_dataset_parser, num_parallel_calls=64)(here).

experimental/inception/inception_v3_old.py/: .batch(batch_size)(here) should be called before .map(parser)(here).

Besides, you need to check the function called in map()(e.g., parser called in .map(parser)) whether to be affected or not to make the changed code work properly. For example, if parser needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
opened by DLPerf 1
Updated three places

Hello,this issue shows the reason that I commit this PR. I sincerely wish my PR will help you.And if you think my PR has a little work,Hoping you could merge it. Thank you,my friend.

opened by DLPerf 0
Performance issues in tpu/models/official/retinanet/dataloader.py

Hello,I found a performance issue in the definition of __call__ , tpu/models/official/retinanet/dataloader.py, dataset = dataset.map(_process_example) was called without num_parallel_calls. I think it will increase the efficiency of your program if you add this.

The same issues also exist in dataset = dataset.map(parser) , dataset = dataset.repeat().map(parser)

Here is the documemtation of tensorflow to support this thing.

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

opened by DLPerf 0
train CIFAR failed

I use ./cifar_trainval.sh to train，but it occurs some problems，why?

Traceback (most recent call last): File "/environment/python/versions/miniconda3-4.7.12/envs/tf1.15/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "/environment/python/versions/miniconda3-4.7.12/envs/tf1.15/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "/environment/python/versions/miniconda3-4.7.12/envs/tf1.15/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found. (0) Internal: Blas GEMM launch failed : a.shape=(128, 64), b.shape=(64, 10), m=128, n=10, k=64 [[{{node resnet/tower_0/fully_connected/dense/MatMul}}]] [[resnet/tower_0/softmax_cross_entropy_loss/assert_broadcastable/is_valid_shape/has_valid_nonscalar_shape/has_invalid_dims/concat/_1549]] (1) Internal: Blas GEMM launch failed : a.shape=(128, 64), b.shape=(64, 10), m=128, n=10, k=64 [[{{node resnet/tower_0/fully_connected/dense/MatMul}}]] 0 successful operations. 0 derived errors ignored.

opened by madoka109 0

Owner

Yin Cui

Research Scientist at Google

GitHub

Pytorch Implementations of large number classical backbone CNNs, data enhancement, torch loss, attention, visualization and some common algorithms.

Torch-template-for-deep-learning Pytorch implementations of some **classical backbone CNNs, data enhancement, torch loss, attention, visualization and

270 Dec 31, 2022

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

32 Sep 21, 2022

An implementation for the loss function proposed in Decoupled Contrastive Loss paper.

Decoupled-Contrastive-Learning This repository is an implementation for the loss function proposed in Decoupled Contrastive Loss paper. Requirements P

71 Dec 4, 2022

Implement of "Training deep neural networks via direct loss minimization" in PyTorch for 0-1 loss

This is the implementation of "Training deep neural networks via direct loss minimization" published at ICML 2016 in PyTorch. The implementation targe

1 Jan 18, 2022

Official implementation for the paper: "Multi-label Classification with Partial Annotations using Class-aware Selective Loss"

Multi-label Classification with Partial Annotations using Class-aware Selective Loss Paper | Pretrained models Official PyTorch Implementation Emanuel

99 Dec 27, 2022

A Pytorch implementation of CVPR 2021 paper "RSG: A Simple but Effective Module for Learning Imbalanced Datasets"

RSG: A Simple but Effective Module for Learning Imbalanced Datasets (CVPR 2021) A Pytorch implementation of our CVPR 2021 paper "RSG: A Simple but Eff

120 Dec 12, 2022

Colar: Effective and Efficient Online Action Detection by Consulting Exemplars, CVPR 2022.

Colar: Effective and Efficient Online Action Detection by Consulting Exemplars This repository is the official implementation of Colar. In this work,

246 Dec 13, 2022

The source code of CVPR 2019 paper "Deep Exemplar-based Video Colorization".

Deep Exemplar-based Video Colorization (Pytorch Implementation) Paper | Pretrained Model | Youtube video ?? | Colab demo Deep Exemplar-based Video Col

253 Dec 27, 2022

A pytorch-version implementation codes of paper: "BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation"

BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation A pytorch-version implementation

11 Oct 8, 2022

Official Pytorch implementation of "Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021)

Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021) Official Pytorch implementation of Unbiased Classification

17 Jan 1, 2023

Class-Balanced Loss Based on Effective Number of Samples. CVPR 2019

Related tags