The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper.

Overview

Intermdiate layer matters - SSL

The official repository for "Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning" paper.

  1. Download the data for the experiments:

The data can be downloaded from kaggle.com. NIH chest-xray dataset: https://www.kaggle.com/nih-chest-xrays/data Breast cancer histopathology dataset: https://www.kaggle.com/paultimothymooney/breast-histopathology-images Diabetic Retinopathy dataset: https://www.kaggle.com/c/diabetic-retinopathy-detection/data

  1. Training of SSL models:

To train the ssl models for moco, moco-mse and moco-btwins, please use 'train_ssl_moco.py', 'train_ssl_moco_mse.py' and 'train_ssl_moco_btwins.py' respectively. The code works for first two datasets. For the diabetic retinopathy dataset, please write a dataloader like "chest_xray_supervised.py" and a datamodule file like "chest_xray_dm.py". Import these files in 'train_ssl_moco.py', 'train_ssl_moco_mse.py' and 'train_ssl_moco_btwins.py' and make necesary changes. The same code can work for the diabetic retinopathy dataset.

  1. Fine tuning the models:

To finetune the models, please use the "fine_tune_moco_chestxray.py" and "fine_tune_moco_hist.py" for NIH chest xray and Breast cancer histopathology data, respectively. For the diabetic retinopathy dataset, please write the code for fine tuning using/similar to "fine_tune_moco_chestxray.py"

  1. Probing the models:

To probe the intermediate layers of the model, please use the "probing_moco_chestxray.py" and "probing_moco_hist.py" for NIH chest xray and Breast cancer histopathology data, respectively. For the diabetic retinopathy dataset, please write the code for probing the intermediate layers using/similar to "probing_moco_chestxray.py"

  1. Feature reuse analysis:

To compute the feature similarity, perform the inference using your model, store the intermediate layer representations and use "CKA.py" for computing the kernel similarity with sigma = 0.8.

You might also like...
Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://arxiv.org/abs/2103.06332).

Hurdles to Progress in Long-form Question Answering This repository contains the official scripts and datasets accompanying our NAACL 2021 paper, "Hur

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Official repository for CVPR21 paper "Deep Stable Learning for Out-Of-Distribution Generalization".

StableNet StableNet is a deep stable learning method for out-of-distribution generalization. This is the official repo for CVPR21 paper "Deep Stable L

CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

TUCH This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] License Software Copyright License fo

Official repository for the paper
Official repository for the paper "Instance-Conditioned GAN"

Official repository for the paper "Instance-Conditioned GAN" by Arantxa Casanova, Marlene Careil, Jakob Verbeek, Michał Drożdżal, Adriana Romero-Soriano.

An official repository for Paper
An official repository for Paper "Uformer: A General U-Shaped Transformer for Image Restoration".

Uformer: A General U-Shaped Transformer for Image Restoration Zhendong Wang, Xiaodong Cun, Jianmin Bao and Jianzhuang Liu Paper: https://arxiv.org/abs

The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022
The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022

DG-TrajGen The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022. Our Meth

Official repository of the paper
Official repository of the paper "A Variational Approximation for Analyzing the Dynamics of Panel Data". Mixed Effect Neural ODE. UAI 2021.

Official repository of the paper (UAI 2021) "A Variational Approximation for Analyzing the Dynamics of Panel Data", Mixed Effect Neural ODE. Panel dat

Comments
  • Many issues with the code / reproducability

    Many issues with the code / reproducability

    Hello,

    I am trying to reproduce some of the results in your paper.

    I am focused for now on the Diabetic Retinopathy results, and in particular the file ssl_mocov2_mse.py

    It seems there are many bugs in this code, including syntax errors and the use or passing of undefined variables, such that it doesn't run.

    Just one example - in your main function here: https://github.com/aakashrkaku/intermdiate_layer_matter_ssl/blob/main/diabetic_retinopathy/ssl_mocov2_mse.py#L521-L544

    You define dm = BaselineDataModule ... but then you are looking for a variable called data_module

    I can try to fix the errors like this (and also make a Dockerised version of the training to make reproducability easier). But if you could provide a script that actually runs and produces your results, that would be very useful!

    Thanks a lot, and thanks for sharing your research code!

    opened by nlml 2
  • How to recreate the diabetic retinopathy dataset?

    How to recreate the diabetic retinopathy dataset?

    Hi there,

    It is unclear from your respository how exactly to recreate the training dataset for Diabetic Retinopathy used in your paper.

    I downloaded the Diabetic Retinopathy dataset from Kaggle, then unzipped it and ran it through the script available here as it seems to me that this is what you have done.

    I then get several different outputs from this script, and it seems the one you have used for training is that output to /jama16-retina-replication/data/eyepacs/bin2

    However, when I run the above-mentioned script, the bin2/validation folder is empty. How exactly did you partition the training set into train and validation?

    Thanks in advance!

    opened by nlml 2
  • About a lot of packages in requirements.txt

    About a lot of packages in requirements.txt

    I tried to review your results, but a lot of packages in requirements.txt were loaded from files which I had no access to. For example, scipy @ file:///tmp/build/80754af9/scipy_1597686620742/work seaborn @ file:///tmp/build/80754af9/seaborn_1600553570093/work SecretStorage @ file:///tmp/build/80754af9/secretstorage_1594419219833/work Would you mind providing the versions of all the packages mentioned (and not mentioned) above? It would be really helpful.

    opened by gskrpb 1
Owner
Aakash Kaku
Enthusiast of using Deep Learning in Medicine and Machine Learning in Finance and Marketing. Master of Business Administration and Data Sciences
Aakash Kaku
Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.

Learning Associative Inference Using Fast Weight Memory This repository contains the offical code for the paper Learning Associative Inference Using F

Imanol Schlag 18 Oct 12, 2022
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

selfcontact This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] It includes the main function

Lea Müller 68 Dec 6, 2022
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

SMPLify-XMC This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] License Software Copyright Lic

Lea Müller 83 Dec 14, 2022
Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers"

Recurrent Fast Weight Programmers This is the official repository containing the code we used to produce the experimental results reported in the pape

IDSIA 36 Nov 15, 2022
Official repository for the paper "Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks"

Easy-To-Hard The official repository for the paper "Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks". Gett

Avi Schwarzschild 52 Sep 8, 2022
Official repository for the CVPR 2021 paper "Learning Feature Aggregation for Deep 3D Morphable Models"

Deep3DMM Official repository for the CVPR 2021 paper Learning Feature Aggregation for Deep 3D Morphable Models. Requirements This code is tested on Py

null 38 Dec 27, 2022
Official repository for the paper, MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Understanding.

MidiBERT-Piano Authors: Yi-Hui (Sophia) Chou, I-Chun (Bronwin) Chen Introduction This is the official repository for the paper, MidiBERT-Piano: Large-

null 137 Dec 15, 2022
Official repository of the paper 'Essentials for Class Incremental Learning'

Essentials for Class Incremental Learning Official repository of the paper 'Essentials for Class Incremental Learning' This Pytorch repository contain

null 33 Nov 27, 2022
This repository is an official implementation of the paper MOTR: End-to-End Multiple-Object Tracking with TRansformer.

MOTR: End-to-End Multiple-Object Tracking with TRansformer This repository is an official implementation of the paper MOTR: End-to-End Multiple-Object

null 348 Jan 7, 2023
Official Repository for the ICCV 2021 paper "PixelSynth: Generating a 3D-Consistent Experience from a Single Image"

PixelSynth: Generating a 3D-Consistent Experience from a Single Image (ICCV 2021) Chris Rockwell, David F. Fouhey, and Justin Johnson [Project Website

Chris Rockwell 95 Nov 22, 2022