Pytorch implementation for "Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets" (ECCV 2020 Spotlight)

Overview

Distribution-Balanced Loss

[Paper]

The implementation of our paper Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets (ECCV2020 Spotlight).

Tong WuQingqiu HuangZiwei LiuYu WangDahua Lin

Requirements

Installation

git clone [email protected]:wutong16/DistributionBalancedLoss.git
cd DistributionBalancedLoss

Quick start

Training

COCO-MLT

python tools/train.py configs/coco/LT_resnet50_pfc_DB.py 

VOC-MLT

python tools/train.py configs/voc/LT_resnet50_pfc_DB.py 

Testing

COCO-MLT

bash tools/dist_test.sh configs/coco/LT_resnet50_pfc_DB.py work_dirs/LT_coco_resnet50_pfc_DB/epoch_8.pth 1

VOC-MLT

bash tools/dist_test.sh configs/voc/LT_resnet50_pfc_DB.py work_dirs/LT_voc_resnet50_pfc_DB/epoch_8.pth 1

Pre-trained models

COCO-MLT

Backbone Total Head Medium Tail Download
ResNet-50 53.55 51.13 57.05 51.06 model

VOC-MLT

Backbone Total Head Medium Tail Download
ResNet-50 78.94 73.22 84.18 79.30 model

Datasets

Use our dataset

The long-tail multi-label datasets we use in the paper are created from MS COCO 2017 and Pascal VOC 2012. Annotations and statistics data resuired when training are saved under ./appendix in this repo.

appendix
  |--coco
    |--longtail2017
      |--class_freq.pkl
      |--class_split.pkl
      |--img_id.pkl
  |--VOCdevkit
    |--longtail2012
      |--class_freq.pkl
      |--class_split.pkl
      |--img_id.pkl

Try your own

You can also create a new long-tailed dataset by downloading the annotations, terse_gt_2017.pkl for COCO and terse_gt_2012.pkl for VOC, from here and move them into the right folders as below.

appendix
  |--coco
    |--longtail2017
      |--terse_gt_2017.pkl
  |--VOCdevkit
    |--longtail2012
      |--terse_gt_2012.pkl

Then run the following command, adjust the parameters as you like to control the distribution.

python tools/create_longtail_dataset.py

To update the corresponding class_freq.pkl files, please refer to def _save_info in .\mllt\datasets\custom.py.

License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@inproceedings{DistributionBalancedLoss,
  title={Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets},
  author={Wu, Tong and Huang, Qingqiu and Liu, Ziwei and Wang, Yu and Lin, Dahua},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2020}
}

TODO

  • Distributed training is not supported currently
  • Evaluation with single GPU is not supported currently
  • test pytorch 0.4.0

Contact

This repo is currently maintained by @wutong16 and @hqqasw

Comments
  • problem of train set and experiment

    problem of train set and experiment

    thanks for your detailed code!

    1. according to your paper and codes, train set only contains 1909 images for coco datasets followed by pareto distribution with max=1200 min=1. i am confused about the params settings, the train set is too samll, why not try to set a larger max param to construct a larger train set.
    2. in table 1, for the ERM settings, are all images sampled uniformly without class aware sampler ? just as most common simply baseline(just like training imagenet except for BCEloss)? I have done some experiments on original coco14 datasets(also a long-tailed distribution), but i find using class-aware sampler get a inferior performance than baseline, 2~3 point below. maybe there is something wrong in my implementation.
    opened by valencebond 4
  • About Class aware sampler

    About Class aware sampler

    https://github.com/wutong16/DistributionBalancedLoss/blob/14460d0acda6a3e77a8a6a5999b9746243cb1535/mllt/datasets/loader/sampler.py#L255 I wonder why this lined was commented and introduce the variance "num_samples_cls". In my opion, this line can accomplish the same function as the lines below, is that right? Or could you please tell why use variance "num_samples_cls", was that for code efficiency? Thanks a lot!

    opened by zhandand 2
  • Reproduction of Distribution-Balanced Loss on VOC Dataset

    Reproduction of Distribution-Balanced Loss on VOC Dataset

    Distribution-Balanced Loss is a very good work, we reproduced the experimental results in coco long tail experiment, but the difference of mAP on VOC dataset is almost 1 point, is there something wrong with our settings, looking forward to your reply~ feab3d95c0ad8b8c89e46613266a741

    opened by limengyang1992 2
  • Lower mAP when training on VOC

    Lower mAP when training on VOC

    Hi, thank you for providing your source code of the paper, Actually, I run you training code of VOC dataset for 3 times, but always get the Total mAP at around 77.7mAP. I notice in your paper "Table 1", the mAP of your "DB-focal" is 78.94. I was wondering if I did something wrong and why this is the case ?

    Below is the log-out information of your training code. Split: head mAP:0.7344 acc:0.8429 micro:0.6096 macro:0.5392 Split:middle mAP:0.8359 acc:0.9375 micro:0.6400 macro:0.6527 Split: tail mAP:0.7651 acc:0.9400 micro:0.5492 macro:0.5305 Split: Total mAP:0.7771 acc:0.9101 micro:0.6024 macro:0.5698

    Thanks in advance.

    opened by Kevinz-code 2
  • ModuleNotFoundError: No module named 'lvis'

    ModuleNotFoundError: No module named 'lvis'

    Traceback (most recent call last): File "tools/train.py", line 11, in from mllt.datasets import build_dataset File "/content/drive/My Drive/DistributionBalancedLoss/mllt/datasets/init.py", line 4, in from .Lvis import LvisDataset File "/content/drive/My Drive/DistributionBalancedLoss/mllt/datasets/Lvis.py", line 2, in from lvis.lvis import LVIS ModuleNotFoundError: No module named 'lvis' when I run python tools/train.py configs/coco/LT_resnet50_pfc_DB.py , this erorr is rasied. I think a lvis file should be existed, however, i can't find it. can you tell how to solve this erorr? thank you!

    opened by taomrzhang 2
  • Dataset?

    Dataset?

    Is the dataset first released by authors? where can I download the dataset you evaluated in your manuscript? Or is there any widely-accepted public dataset for multi-label long-tailed classification task?

    opened by PyJulie 1
  • Support multi-GPU training?

    Support multi-GPU training?

    Hi, I tried to run python tools/train.py configs/coco/LT_resnet50_pfc_DB.py --gpus 2

    and met some bugs:

    File "/data2/cjq/DistributionBalancedLoss/mllt/models/losses/resample_loss.py", line 164, in rebalance_weight repeat_rate = torch.sum( gt_labels.float() * self.freq_inv, dim=1, keepdim=True) RuntimeError: expected device cuda:1 and dtype Float but got device cuda:0 and dtype Float

    Does this code support multi-GPU training? Have you ever tried, or I should fix these bugs by myself?

    Thanks for you code !

    opened by chen-judge 1
  • formulation error in Eq 7

    formulation error in Eq 7

    in Eq7, the for positive samples, zik in exponent should be -zik,and for negative samples,-λ *zik should be λ *zik, which is consistent with formulation in Eq5.

    please correct me if there is something wrong. thanks~

    opened by valencebond 1
  • About  code implementation of eq7

    About code implementation of eq7

    Hey thanks for the code 😄 but have some questions!!

    https://github.com/wutong16/DistributionBalancedLoss/blob/14460d0acda6a3e77a8a6a5999b9746243cb1535/mllt/models/losses/resample_loss.py#L158-L160

    Isn't this inconsistent with eq7
    Screenshot 2022-10-28 at 2 25 03 AM

    1. Like line 159 is dealing with the exp power in eq7 right? (Line 159) why are we directly multiplying it with labels, that anyhow occurs when we pass it to self.cls_criterion right? (same goes for weight)

    2. Bit confused in name https://github.com/wutong16/DistributionBalancedLoss/blob/14460d0acda6a3e77a8a6a5999b9746243cb1535/mllt/models/losses/resample_loss.py#L113 Is cls_score here (logits passed through sigmoid/softmax) or just pure raw classifier outputs?, because even though we call it cls_score it's passed the to the same F.binary_cross_entropy_with_logits_loss

    opened by saishkomalla 0
  • FileNotFoundError: [Errno 2] No such file or directory: '/mnt/SSD/det/coco/annotations/instances_train2017.json'

    FileNotFoundError: [Errno 2] No such file or directory: '/mnt/SSD/det/coco/annotations/instances_train2017.json'

    Traceback (most recent call last): File "tools/train.py", line 108, in main() File "tools/train.py", line 85, in main train_dataset = build_dataset(cfg.data.train) File "/root/autodl-tmp/DB_loss/DistributionBalancedLoss/mllt/datasets/builder.py", line 36, in build_dataset dataset = build_from_cfg(cfg, DATASETS) File "/root/autodl-tmp/DB_loss/DistributionBalancedLoss/mllt/utils/registry.py", line 74, in build_from_cfg return obj_type(**args) File "/root/autodl-tmp/DB_loss/DistributionBalancedLoss/mllt/datasets/coco.py", line 27, in init super(CocoDataset, self).init(**kwargs) File "/root/autodl-tmp/DB_loss/DistributionBalancedLoss/mllt/datasets/custom.py", line 59, in init self.img_infos = self.load_annotations(ann_file, LT_ann_file) File "/root/autodl-tmp/DB_loss/DistributionBalancedLoss/mllt/datasets/coco.py", line 32, in load_annotations self.coco = COCO(ann_file) File "/root/miniconda3/envs/blast/lib/python3.7/site-packages/pycocotools-2.0-py3.7-linux-x86_64.egg/pycocotools/coco.py", line 84, in init dataset = json.load(open(annotation_file, 'r')) FileNotFoundError: [Errno 2] No such file or directory: '/mnt/SSD/det/coco/annotations/instances_train2017.json'

    When I run python tools/train.py configs/coco/LT_resnet50_pfc_DB.py ,this error was rasied. I think this is because the program did not find the location of coco dataset. But I'm not sure. Can you tell me how to solve this error? Thank you!

    opened by AL-Yan 0
  • bug: args.gpus is missing

    bug: args.gpus is missing

    args.gpus is missing https://github.com/wutong16/DistributionBalancedLoss/blob/a3ecaa9021a920fcce9fdafbd7d83b51bf526af8/tools/train.py#L59

    maybe add parser.add_argument('--gpus', default=1) in the parse_args()

    opened by VamosC 0
Owner
Tong WU
Tong WU
SNE-RoadSeg in PyTorch, ECCV 2020

SNE-RoadSeg Introduction This is the official PyTorch implementation of SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentati

null 242 Dec 20, 2022
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

五维空间 140 Nov 23, 2022
[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

Stable Head Pose Estimation and Landmark Regression via 3D Dense Face Reconstruction Reimplementation of (ECCV 2020) Towards Fast, Accurate and Stable

Remilia Scarlet 221 Dec 30, 2022
1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for Tracking

Instead, two models for appearance modeling are included, together with the open-source BAGS model and the full set of code for inference. With this code, you can achieve around mAP@23 with TAO test set (based on our estimation).

null 79 Oct 8, 2022
Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Causality In Traffic Accident (Under Construction) Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020) Overview Data Prepa

Tackgeun 21 Nov 20, 2022
git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction Code for the ECCV 2020 paper by Yiming Qian and Yasutaka Furukawa Getting

null 37 Dec 4, 2022
Code for our paper at ECCV 2020: Post-Training Piecewise Linear Quantization for Deep Neural Networks

PWLQ Updates 2020/07/16 - We are working on getting permission from our institution to release our source code. We will release it once we are granted

null 54 Dec 15, 2022
dataset for ECCV 2020 "Motion Capture from Internet Videos"

Motion Capture from Internet Videos Motion Capture from Internet Videos Junting Dong*, Qing Shuai*, Yuanqing Zhang, Xian Liu, Xiaowei Zhou, Hujun Bao

ZJU3DV 98 Dec 7, 2022
Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Adversarial Training Against Location-Optimized Adversarial Patches arXiv | Paper | Code | Video | Slides Code for the paper: Sukrut Rao, David Stutz,

Sukrut Rao 32 Dec 13, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 42 Nov 24, 2022
Code for the paper "Improving Vision-and-Language Navigation with Image-Text Pairs from the Web" (ECCV 2020)

Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh

Arjun Majumdar 44 Dec 14, 2022
Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Contact and Human Dynamics from Monocular Video This is the official implementation for the ECCV 2020 spotlight paper by Davis Rempe, Leonidas J. Guib

Davis Rempe 207 Jan 5, 2023
[ECCV 2020] Gradient-Induced Co-Saliency Detection

Gradient-Induced Co-Saliency Detection Zhao Zhang*, Wenda Jin*, Jun Xu, Ming-Ming Cheng ⭐ Project Home » The official repo of the ECCV 2020 paper Grad

Zhao Zhang 35 Nov 25, 2022
Code for Towards Streaming Perception (ECCV 2020) :car:

sAP — Code for Towards Streaming Perception ECCV Best Paper Honorable Mention Award Feb 2021: Announcing the Streaming Perception Challenge (CVPR 2021

Martin Li 85 Dec 22, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 29 Dec 28, 2022
Sign Language Translation with Transformers (COLING'2020, ECCV'20 SLRTP Workshop)

transformer-slt This repository gathers data and code supporting the experiments in the paper Better Sign Language Translation with STMC-Transformer.

Kayo Yin 107 Dec 27, 2022
Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Progressive Transformers for End-to-End Sign Language Production Source code for "Progressive Transformers for End-to-End Sign Language Production" (B

null 58 Dec 21, 2022
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to your questions. This repo is almost the same with Another-Version, and you can also refer to that version.

CVSM Group -  email: czhu@bupt.edu.cn 84 Dec 12, 2022
Boundary-preserving Mask R-CNN (ECCV 2020)

BMaskR-CNN This code is developed on Detectron2 Boundary-preserving Mask R-CNN ECCV 2020 Tianheng Cheng, Xinggang Wang, Lichao Huang, Wenyu Liu Video

Hust Visual Learning Team 178 Nov 28, 2022