Learning to See by Looking at Noise

Related tags

Deep Learning learning_with_noise
Overview

Learning to See by Looking at Noise

This is the official implementation of Learning to See by Looking at Noise.

In this work, we investigate a suite of image generation models that produce images from simple random processes. These are then used as training data for a visual representation learner with a contrastive loss. We study two types of noise processes, statistical image models and deep generative models under different random initializations.

[Project page] [Paper] [arXiv]

Requirements

This version of code has been tested with Python 3.7.7 and pytorch 1.6.0. Other versions of pytorch are likely to work out of the box. The contrastive training requires two GPU's with at least 12GB of memory for the small scale experiments, while the large scale experiments require the same computation resources as the facebookresearch implementation of MoCo.

To use this repo, first clone it and correct the permissions for the scripts with:

git clone https://github.com/mbaradad/learning_with_noise
cd learning_with_noise
chmod 755 -R scripts

To install all the requirements, simply do:

pip intall -r requirements.txt

Small scale experiments

To download all the datasets, first run:

./scripts/download_datasets/download_small_scale_datasets.sh

Then you can launch the contrastive training for all the small scale experiments with:

./scripts/train_align_uniform/main.sh <GPU_ID_0> <GPU_ID_1>

If you just want to test the linear evaluation of the models (or do something else with them), you can directly download our pretrained encoders with:

./scripts/download_pretrained_models/download_all_alexnet_encoders.sh

Finally, you can evaluate the linear performance with imagenet100 as:

./scripts/train_align_uniform/linear_eval.sh <path-to-imagenet100> <GPU_ID>

Where is the path to the imagenet100 dataset dir, which should contain two dirs (train and val) each with the train and val samples respectively for the 100 imagenet100 classes. If you have imagenet1k, you can generate imagenet100 using the following command (which will create simlyncs to your imagenet1k dir):

./scripts/datasets/generate_imagenet100.sh <path-to-imagenet1k> <path-to-imagenet100>

Large scale experiments

Datasets and encoders will be be released soon!

Data generation

Scripts to generate the datasets will be released soon!

You might also like...
Neural Magic Eye: Learning to See and Understand the Scene Behind an Autostereogram, arXiv:2012.15692.
Neural Magic Eye: Learning to See and Understand the Scene Behind an Autostereogram, arXiv:2012.15692.

Neural Magic Eye Preprint | Project Page | Colab Runtime Official PyTorch implementation of the preprint paper "NeuralMagicEye: Learning to See and Un

Zhengxia Zou 56 Jul 15, 2022
DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

Facebook Research 145 Dec 30, 2022
NR-GAN: Noise Robust Generative Adversarial Networks
NR-GAN: Noise Robust Generative Adversarial Networks

NR-GAN: Noise Robust Generative Adversarial Networks (CVPR 2020) This repository provides PyTorch implementation for noise robust GAN (NR-GAN). NR-GAN

Takuhiro Kaneko 59 Dec 11, 2022
The Noise Contrastive Estimation for softmax output written in Pytorch
The Noise Contrastive Estimation for softmax output written in Pytorch

An NCE implementation in pytorch About NCE Noise Contrastive Estimation (NCE) is an approximation method that is used to work around the huge computat

Kaiyu Shi 287 Nov 25, 2022
PyTorch implementation of "A Two-Stage End-to-End System for Speech-in-Noise Hearing Aid Processing"

Implementation of the Sheffield entry for the first Clarity enhancement challenge (CEC1) This repository contains the PyTorch implementation of "A Two

null 10 Aug 19, 2022
Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel
Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel

Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel This repository is the official PyTorch implementation of BSRDM w

Zongsheng Yue 69 Jan 5, 2023
Riemann Noise Injection With PyTorch

Riemann Noise Injection - PyTorch A module for modeling GAN noise injection based on Riemann geometry, as described in Ruili Feng, Deli Zhao, and Zhen

null 2 May 27, 2022
Self-Supervised Methods for Noise-Removal

SSMNR | Self-Supervised Methods for Noise Removal Image denoising is the task of removing noise from an image, which can be formulated as the task of

null 1 Jan 16, 2022
The trained model and denoising example for paper : Cardiopulmonary Auscultation Enhancement with a Two-Stage Noise Cancellation Approach

The trained model and denoising example for paper : Cardiopulmonary Auscultation Enhancement with a Two-Stage Noise Cancellation Approach

ycj_project 1 Jan 18, 2022
Comments
  • VTAB evaluation

    VTAB evaluation

    Hi, Nice work! Thanks for sharing the source code. I wonder how should we evaluate the pre-trained model on VTAB datasets? My question is should the pre-trained model be fixed and evaluated with linear classification or is it finetuned. If you can also release that part of the codes, it will be great. Thanks a lot!

    opened by yangyu12 3
  • Will the Data Be Made Available?

    Will the Data Be Made Available?

    Will a download to the samples themselves be made available? This would be more convenient than regenerating the data. Likewise the AlexNets trained on these datasets would be helpful too.

    opened by hendrycks 3
  • downloading encoders

    downloading encoders

    Hi, I get this error when I run ./scripts/download_pretrained_models/download_large_scale_encoders.sh:

    Downloading pretrained model for dataset dead_leaves-mixed
--2022-05-01 17:40:19--  http://data.csail.mit.edu/noiselearning/encoders/large_scale/dead_leaves-mixed/checkpoint_0199.pth.tar
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 369874784 (353M) [application/octet-stream]
Saving to: ‘encoders/large_scale/dead_leaves-mixed/checkpoint_0199.pth.tar’

encoders/large_scale/dead_leaves-mixed/ 100%[=============================================================================>] 352.74M  75.4MB/s    in 4.7s

2022-05-01 17:40:24 (75.4 MB/s) - ‘encoders/large_scale/dead_leaves-mixed/checkpoint_0199.pth.tar’ saved [369874784/369874784]

Downloading pretrained model for dataset feature_vis-dead_leaves
--2022-05-01 17:40:24--  http://data.csail.mit.edu/noiselearning/encoders/large_scale/feature_vis-dead_leaves/checkpoint_0199.pth.tar
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 369735063 (353M) [application/octet-stream]
Saving to: ‘encoders/large_scale/feature_vis-dead_leaves/checkpoint_0199.pth.tar’

encoders/large_scale/feature_vis-dead_l 100%[=============================================================================>] 352.61M  84.1MB/s    in 4.2s

2022-05-01 17:40:29 (84.1 MB/s) - ‘encoders/large_scale/feature_vis-dead_leaves/checkpoint_0199.pth.tar’ saved [369735063/369735063]

Downloading pretrained model for dataset stat-spectrum_color_wmm
--2022-05-01 17:40:29--  http://data.csail.mit.edu/noiselearning/encoders/large_scale/stat-spectrum_color_wmm/checkpoint_0199.pth.tar
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 369735063 (353M) [application/octet-stream]
Saving to: ‘encoders/large_scale/stat-spectrum_color_wmm/checkpoint_0199.pth.tar’

encoders/large_scale/stat-spectrum_colo 100%[=============================================================================>] 352.61M  77.4MB/s    in 4.5s

2022-05-01 17:40:33 (78.4 MB/s) - ‘encoders/large_scale/stat-spectrum_color_wmm/checkpoint_0199.pth.tar’ saved [369735063/369735063]

Downloading pretrained model for dataset stylegan-oriented
--2022-05-01 17:40:34--  http://data.csail.mit.edu/noiselearning/encoders/large_scale/stylegan-oriented/checkpoint_0199.pth.tar
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 369735063 (353M) [application/octet-stream]
Saving to: ‘encoders/large_scale/stylegan-oriented/checkpoint_0199.pth.tar’

encoders/large_scale/stylegan-oriented/ 100%[=============================================================================>] 352.61M  83.1MB/s    in 4.3s

2022-05-01 17:40:38 (81.4 MB/s) - ‘encoders/large_scale/stylegan-oriented/checkpoint_0199.pth.tar’ saved [369735063/369735063]

Downloading pretrained model for dataset mixed-4
--2022-05-01 17:40:38--  http://data.csail.mit.edu/noiselearning/encoders/large_scale/mixed-4/checkpoint_0199.pth.tar
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2022-05-01 17:40:38 ERROR 404: Not Found.
    opened by harshays 1
  • Some python modules missing?

    Some python modules missing?

    Hi,

    Thanks for releasing great work. It seems that some python modules are missing in your repo: my_python_utils, my_python_utils and maybe others. So scripts python stylegan/generate_dataset.py --small-scale --network_type original do not work.

    opened by bknyaz 1
Owner
Manel Baradad Jurjo
PhD candidate at CSAIL (MIT)
Manel Baradad Jurjo
GitHub
PyTorch implementation of SmoothGrad: removing noise by adding noise.

SmoothGrad implementation in PyTorch PyTorch implementation of SmoothGrad: removing noise by adding noise. Vanilla Gradients SmoothGrad Guided backpro

SSKH 143 Jan 5, 2023
Official implementation of "Open-set Label Noise Can Improve Robustness Against Inherent Label Noise" (NeurIPS 2021)

Open-set Label Noise Can Improve Robustness Against Inherent Label Noise NeurIPS 2021: This repository is the official implementation of ODNL. Require

Hongxin Wei 12 Dec 7, 2022
Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Tom-R.T.Kvalvaag 2 Dec 17, 2021
git《Beta R-CNN: Looking into Pedestrian Detection from Another Perspective》(NeurIPS 2020) GitHub:[fig3]

Beta R-CNN: Looking into Pedestrian Detection from Another Perspective This is the pytorch implementation of our paper "[Beta R-CNN: Looking into Pede

null 35 Sep 8, 2021
《Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching》(CVPR 2020)

This contains the codes for cross-view geo-localization method described in: Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching, CVPR2020.

null 41 Oct 27, 2022
Angular & Electron desktop UI framework. Angular components for native looking and behaving macOS desktop UI (Electron/Web)

Angular Desktop UI This is a collection for native desktop like user interface components in Angular, especially useful for Electron apps. It starts w

Marc J. Schmidt 49 Dec 22, 2022
PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)

2021-CVPR-MvCLN This repo contains the code and data of the following paper accepted by CVPR 2021 Partially View-aligned Representation Learning with

XLearning Group 33 Nov 1, 2022
[EMNLP 2021] Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training

RoSTER The source code used for Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training, p

Yu Meng 60 Dec 30, 2022
Code of our paper "Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning"

CCOP Code of our paper Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning Requirement Install OpenSelfSup Install Detectron2

Chenhongyi Yang 21 Dec 13, 2022
Official implementation of our paper "Learning to Bootstrap for Combating Label Noise"

Learning to Bootstrap for Combating Label Noise This repo is the official implementation of our paper "Learning to Bootstrap for Combating Label Noise

null 21 Apr 9, 2022