[ICLR 2021] Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Kaidi Cao

Last update: Oct 20, 2022

Related tags

Deep Learning HAR

Overview

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon, Tengyu Ma

Dependency

The code is built with following libraries:

PyTorch 1.8

Training

The whole HAR training pipeline can be done in the following three steps:

To estimate the statistics through a pretrain step

python cifar_hetero_est.py --mislabel_type hetero --gpu 0 --split 0

To calculate the weights for regularization

python weight_est.py --statspath ./log/estimate_cifar10_resnet32_hetero_0.5_0_example/stats0.pkl

Finally train a model from the scratch

python cifar_train.py --dataset cifar10  --rand-number 0 --mislabel_type hetero --imb_type None --gpu 0 --reg_weight 10 --exp_str example --reg_path ./data/cifar10_example_weights.npy

Reference

If you find our paper and repo useful, please cite as

@inproceedings{cao2020heteroskedastic,
  title={Heteroskedastic and imbalanced deep learning with adaptive regularization},
  author={Cao, Kaidi and Chen, Yining and Lu, Junwei and Arechiga, Nikos and Gaidon, Adrien and Ma, Tengyu},
  booktitle={International Conference on Learning Representations}, 
  year={2021} 
}

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

34 Nov 16, 2022

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Neural Attention Distillation This is an implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep

84 Jan 4, 2023

Adaout is a practical and flexible regularization method with high generalization and interpretability

Adaout Adaout is a practical and flexible regularization method with high generalization and interpretability. Requirements python 3.6 (Anaconda versi

1 Feb 9, 2022

The repo contains the code of the ACL2020 paper `Dice Loss for Data-imbalanced NLP Tasks`

Dice Loss for NLP Tasks This repository contains code for Dice Loss for Data-imbalanced NLP Tasks at ACL2020. Setup Install Package Dependencies The c

223 Dec 17, 2022

MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data

This repository is the official PyTorch implementation of Meta-Balance. Find the paper on arxiv MetaBalance: High-Performance Neural Networks for Clas

20 Oct 18, 2021

Official implementation of Influence-balanced Loss for Imbalanced Visual Classification in PyTorch.

70 Jan 3, 2023

Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness

Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness Code for Paper "Imbalanced Gradients: A Subtle Cause of Overestimated Adv

11 Nov 30, 2022

BESS: Balanced Evolutionary Semi-Stacking for Disease Detection via Partially Labeled Imbalanced Tongue Data

Balanced-Evolutionary-Semi-Stacking Code for the paper ''BESS: Balanced Evolutionary Semi-Stacking for Disease Detection via Partially Labeled Imbalan

0 Jan 16, 2022

Source code, datasets and trained models for the paper Learning Advanced Mathematical Computations from Examples (ICLR 2021), by François Charton, Amaury Hayat (ENPC-Rutgers) and Guillaume Lample

Maths from examples - Learning advanced mathematical computations from examples This is the source code and data sets relevant to the paper Learning a

171 Nov 23, 2022

Comments

Question about How to conduct experiments on imbalance datasets?

In this repo, "heteroskedastic_cifar.py" contains the code for adding noisy labels in cifar datasets but only has "step"-type imbalance. I tried to use "imbalance_cifar.py" from LDAM to produce experiments on long-tailed datasets. But the training seems abnormal: when I estimate the statistics through a pre-train step using "cifar_hetero_est.py" with IF-100 CIFAR10, the accuracy has achieved 81% during the pre-training. Could you explain why it happen?

opened by hongxin001 0

[ICLR 2021] Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Related tags

Overview

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Dependency

Training

Reference

You might also like...

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Adaout is a practical and flexible regularization method with high generalization and interpretability

The repo contains the code of the ACL2020 paper `Dice Loss for Data-imbalanced NLP Tasks`

MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data

Official implementation of Influence-balanced Loss for Imbalanced Visual Classification in PyTorch.

Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness

BESS: Balanced Evolutionary Semi-Stacking for Disease Detection via Partially Labeled Imbalanced Tongue Data

Source code, datasets and trained models for the paper Learning Advanced Mathematical Computations from Examples (ICLR 2021), by François Charton, Amaury Hayat (ENPC-Rutgers) and Guillaume Lample

Comments

Question about How to conduct experiments on imbalance datasets?

Owner

Kaidi Cao

[ICML 2021, Long Talk] Delving into Deep Imbalanced Regression

A Pytorch implementation of CVPR 2021 paper "RSG: A Simple but Effective Module for Learning Imbalanced Datasets"

[NeurIPS 2021] “Improving Contrastive Learning on Imbalanced Data via Open-World Sampling”,

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

PyTorch framework, for reproducing experiments from the paper Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks

Tensorflow 2 implementation of the paper: Learning and Evaluating Representations for Deep One-class Classification published at ICLR 2021

Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (ICCV 2021)

Code for CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

A (PyTorch) imbalanced dataset sampler for oversampling low frequent classes and undersampling high frequent ones.

[ICLR 2021] "CPT: Efficient Deep Neural Network Training via Cyclic Precision" by Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, Yingyan Lin