PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

Last update: Nov 23, 2022

Overview

Contrast to Divide: self-supervised pre-training for learning with noisy labels

This is an official implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels". The code is based on DivideMix implementation.

Results

Following tables summarize main resutls of the paper:

CIFAR-10:

CIFAR-100:

Clothing1M:

mini-WebVision:

Running the code

First you need to install dependencies by running pip install -r requirements.txt.

You can download pretrained self-supervised models from Google Drive. Alternatively, you can train them by yourself, using SimCLR implementation. Put them into ./pretrained folder.

Then you can run the code for CIFAR

python3 main_cifar.py --r 0.8 --lambda_u 500 --dataset cifar100 --p_threshold 0.03 --data_path ./cifar-100 --experiment-name simclr_resnet18 --method selfsup --net resnet50

for Clothing1M

python3 main_clothing1M.py --data_path /path/to/clothing1m --experiment-name selfsup --method selfsup --p_threshold 0.7 --warmup 5 --num_epochs 120

or for mini-WebVision

python3 Train_webvision.py --p_threshold 0.03 --num_class 50 --data_path /path/to/webvision --imagenet_data_path /path/to/imagenet --method selfsup```

To run C2D with ELR+ just use the self-suprevised pretrained models with the original code.

License

This project is licensed under the terms of the MIT license.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

This repository holds NVIDIA-maintained utilities to streamline mixed precision and distributed training in Pytorch. Some of the code here will be included in upstream Pytorch eventually. The intention of Apex is to make up-to-date utilities available to users as quickly as possible.

6.9k Jan 3, 2023

Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

PyTorch-LIT PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices. With

157 Dec 11, 2022

Comments

tabular data/ noisy instances
Hi, thanks for sharing your implementation. I have two questions about it:

Does it also work on tabular data?

Is it possible to identify the noisy instances (return the noisy IDs or the clean set)?

Thanks!
opened by nazaretl 1
Lambda_u for CIFAR-100 on 40 asym noise

Hi, I haven't been able to find which hyper-parameters you use to train on CIFAR-100 with 40% asymmetric noise. Can you please tell me?

Thank you!

P.S: Awesome work!

opened by aldakata 1
About the self-supervised model on cifar10

Hi, I have trained self-supervised model on cifar10 with PreAct-ResNet18 and ResNet18. On the 90% noise rate, I achieve the performance about 93.46 with ResNet18, but get a worse result about 92.8 with PreAct-ResNet18. However, the performance with PreAct-ResNet18 in the paper is 93.57. And the available self-supervised model for cifar10 in Google Drive is ResNet18, I achieve the performance about 93.54 by using this model. I notice that the performance with PreAct-ResNet18 in the paper is actually more similar to ResNet18. Is there any mistake? Or how can I reproduce the performance with PreAct-ResNet18?

opened by luan-g 0

PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

Related tags

Overview

Contrast to Divide: self-supervised pre-training for learning with noisy labels

Results

Running the code

License

You might also like...

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

A bunch of random PyTorch models using PyTorch's C++ frontend

PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

Comments

tabular data/ noisy instances

Lambda_u for CIFAR-100 on 40 asym noise

About the self-supervised model on cifar10

Owner

An essential implementation of BYOL in PyTorch + PyTorch Lightning

RealFormer-Pytorch Implementation of RealFormer using pytorch

A PyTorch implementation of the paper Mixup: Beyond Empirical Risk Minimization in PyTorch

A pytorch implementation of Pytorch-Sketch-RNN

PyTorch implementation of Advantage async actor-critic Algorithms (A3C) in PyTorch

Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

RetinaNet-PyTorch - A RetinaNet Pytorch Implementation on remote sensing images and has the similar mAP result with RetinaNet in MMdetection

RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, and DVC.