NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Pytorch implementation for noisy labels).

Last update: Jan 3, 2023

Overview

Meta-Weight-Net

NeurIPS'19: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (Official Pytorch implementation for noisy labels). The implementation of class imbalance is available at https://github.com/xjtushujun/Meta-weight-net_class-imbalance.

================================================================================================================================================================

This is the code for the paper: Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, Deyu Meng* To be presented at NeurIPS 2019.

If you find this code useful in your research then please cite

@inproceedings{han2018coteaching,
  title={Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting},
  author={Shu, Jun and Xie, Qi and Yi, Lixuan and Zhao, Qian and Zhou, Sanping and Xu, Zongben and Meng, Deyu},
  booktitle={NeurIPS},
  year={2019}
}

Setups

The requiring environment is as bellow:

Linux
Python 3+
PyTorch 0.4.0
Torchvision 0.2.0

Running Meta-Weight-Net on benchmark datasets (CIFAR-10 and CIFAR-100).

Here is an example:

python train_WRN-28-10_Meta_PGC.py --dataset cifar10 --corruption_type unif(flip2) --corruption_prob 0.6

The default network structure is WRN-28-10, if you want to train with ResNet32 model, please reset the learning rate delay policy.

A stable version is relased.

python MW-Net.py --dataset cifar10 --corruption_type unif(flip2) --corruption_prob 0.6

Important Updating Version

The new code on github (https://github.com/ShiYunyi/Meta-Weight-Net_Code-Optimization) has implemented the MW-Net based on the newest pytorch and torchvision version. It rewrites an optimizer to assign non leaf node tensors to model parameters. Thus it does not need to rewrite the nn.Module as this version does. Very thanks for Shi Yunyi ([email protected])!

Acknowledgements

We thank the Pytorch implementation on glc(https://github.com/mmazeika/glc) and learning-to-reweight-examples(https://github.com/danieltan07/learning-to-reweight-examples).

Contact: Jun Shu ([email protected]); Deyu Meng([email protected]).

You might also like...

Implements a polyglot REPL which supports multiple languages and shared meta-object protocol scope between REPLs.

MetaCall Polyglot REPL Description This repository implements a Polyglot REPL which shares the state of the meta-object protocol between the REPLs. Us

10 Dec 28, 2022

Python library and cli util for https://www.zerochan.net/

Zerochan Library for Zerochan.net with pics parsing and downloader included! Features CLI utility for pics downloading from zerochan.net Library for c

10 Oct 11, 2022

Cross-platform .NET Core pre-commit hooks

dotnet-core-pre-commit Cross-platform .NET Core pre-commit hooks How to use Add this to your .pre-commit-config.yaml - repo: https://github.com/juan

5 Jul 20, 2021

Заглушки .NET библиотек для IronPython

Код репозитория основан на ironpython-stubs. Выражаю gtalarico бесконечную благодарность за вклад в развитие сообщества разработчиков скриптов и плаги

12 Nov 23, 2022

A Python package to request and process seismic waveform data from Hi-net.

HinetPy is a Python package to simplify tedious data request, download and format conversion tasks related to NIED Hi-net. NIED Hi-net | Source Code |

65 Dec 9, 2022

$Implementation of the Angular Spectrum method in Python to simulate Diffraction Patterns$

Implementation of the Angular Spectrum method in Python to simulate Diffraction Patterns

Diffraction Simulations - Angular Spectrum Method Implementation of the Angular Spectrum method in Python to simulate Diffraction Patterns with arbitr

276 Dec 30, 2022

Minimalist BERT implementation assignment for CS11-747

minbert Assignment by Zhengbao Jiang, Shuyan Zhou, and Ritam Dutt This is an exercise in developing a minimalist version of BERT, part of Carnegie Mel

51 Jan 3, 2023

Python implementation of the ASFLIP advection method

This is a python implementation of the ASFLIP advection method . We would like to hear from you if you appreciate this work.

133 Nov 13, 2022

A Gura parser implementation for Python

Gura parser This repository contains the implementation of a Gura format parser in Python. Installation pip install gura-parser Usage import gura gur

19 Jan 25, 2022

Comments

The accuracy of BaseModel is 88.5 when the noise rate is 0.4?

I trained a WRN-28-10 network on cifar-10 with noise rate of 0.4 under uniform noise following the setting in the paper for a total of 40 epochs, but the accuracy of BaseModel is 88.5, which is really high compared with the results in Table 2. I don't know what the problem is.

opened by lzm10214066 4
About the effectiveness

Thank you for your excellent work!

Here, we raised some questions about the effectiveness (time & memory) of Meta-Weight-Net：

Compared to Learn-to-Reweight (Ren, 2017), how about the cost of running time and GPU memory per training step?

Can the training process's time efficiency be improved by updating Meta-Weight-Net every several steps (rather than updating every step)? Will this affect model's performance?

Is it possible to achieve multi-GPU parallelism (based on Pytorch)?

Thanks very much~

opened by SunSiShining 2
tabular data/ noisy instances
Hi, thanks for sharing your implementation. I have two questions about it:

Does it also work on tabular data?

Is it possible to identify the noisy instances (return the noisy IDs or the clean set)?

Thanks!
opened by nazaretl 0
About the details of learning rate

There is a sentence in the appendix: "With batch normalization, we effectively cancel the learning rate of Meta-Weight-Net, and it works well with a fixed learning rate. "

I'm not sure what it is about. Would you please give an explanation in detail? Does it mean we don't need to fine-tune the learning rate of meta networks because of BN?

opened by hongxin001 1