Demystifying How Self-Supervised Features Improve Training from Noisy Labels

Last update: Oct 14, 2022

Related tags

Overview

Demystifying How Self-Supervised Features Improve Training from Noisy Labels

This code is a PyTorch implementation of the paper "[Demystifying How Self-Supervised Features Improve Training from Noisy Labels]". The code is run on the Tesla V-100.

Prerequisites

Python 3.8.5

PyTorch 1.7.1

Torchvision 0.9.0

Guideline

Downloading dataset:

Download the dataset from http://www.cs.toronto.edu/~kriz/cifar.html Put the dataset on data/

Get a SSL pre-trained model:

Run command below:

python main.py --batch_size 512 --epochs 1000

Or download the pre-trained model from "this url" and put the pre-trained model in results/

CE with fixed encoder on symm. label noise:

Run command below:

python CE.py --simclr_pretrain --finetune_fc_only --noise_type symmetric --noise_rate 0.6 --epochs 150

CE with fixed encoder on instance label noise (with down-sampling):

Run command below:

python CE.py --simclr_pretrain --down_sample --finetune_fc_only --noise_type instance --noise_rate 0.6 --epochs 150

CE with regularizer on symm. label noise (random initialization):

Run command below:

python CE_Reg.py --reg rkd_dis --noise_type symmetric --noise_rate 0.6 --epochs 150

References

The code of SSL pre-training is based on https://github.com/leftthomas/SimCLR

The Self-Supervised Learner can be used to train a classifier with fewer labeled examples needed using self-supervised learning.

Published by SpaceML • About SpaceML • Quick Colab Example Self-Supervised Learner The Self-Supervised Learner can be used to train a classifier with

92 Nov 30, 2022

[CVPR 2021] "The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models" Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Codes for this paper The Lottery Tickets Hypo

59 Dec 28, 2022

Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

UniSpeech The family of UniSpeech: UniSpeech (ICML 2021): Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR UniSpeech-

282 Jan 9, 2023

Experiments on Flood Segmentation on Sentinel-1 SAR Imagery with Cyclical Pseudo Labeling and Noisy Student Training

Flood Detection Challenge This repository contains code for our submission to the ETCI 2021 Competition on Flood Detection (Winning Solution #2). Acco

108 Dec 28, 2022

Training a deep learning model on the noisy CIFAR dataset

Training-a-deep-learning-model-on-the-noisy-CIFAR-dataset This repository contai

1 Jun 14, 2022

This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels].

CGPN This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels]. Req

10 Sep 12, 2022

[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Using Unreliable Pseudo Labels Official PyTorch implementation of Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels, CVPR 2022. Ple

268 Dec 24, 2022

Implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

SemCo The official pytorch implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

42 Nov 14, 2022

The official implementation of the paper, "SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning"

SubTab: Author: Talip Ucar ([email protected]) The official implementation of the paper, SubTab: Subsetting Features of Tabular Data for Self-Supervis

98 Dec 29, 2022

Demystifying How Self-Supervised Features Improve Training from Noisy Labels

Related tags

Overview

Demystifying How Self-Supervised Features Improve Training from Noisy Labels

Prerequisites

Guideline

Downloading dataset:

Get a SSL pre-trained model:

CE with fixed encoder on symm. label noise:

CE with fixed encoder on instance label noise (with down-sampling):

CE with regularizer on symm. label noise (random initialization):

References

You might also like...

The Self-Supervised Learner can be used to train a classifier with fewer labeled examples needed using self-supervised learning.

[CVPR 2021] "The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models" Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang

Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

Experiments on Flood Segmentation on Sentinel-1 SAR Imagery with Cyclical Pseudo Labeling and Noisy Student Training

Training a deep learning model on the noisy CIFAR dataset

This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels].

[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

The official implementation of the paper, "SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning"

Owner

REAL@UCSC

Large-Scale Pre-training for Person Re-identification with Noisy Labels (LUPerson-NL)

A curated (most recent) list of resources for Learning with Noisy Labels

A GOOD REPRESENTATION DETECTS NOISY LABELS

NeurIPS 2021, "Fine Samples for Learning with Noisy Labels"

Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels

Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

A Light CNN for Deep Face Representation with Noisy Labels

Sample Prior Guided Robust Model Learning to Suppress Noisy Labels

Official code for paper "Demystifying Local Vision Transformer: Sparse Connectivity, Weight Sharing, and Dynamic Weight"

The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We significantly improve the systematic generalization of transformer models on a variety of datasets using simple tricks and careful considerations.