PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

opcrisis

Last update: Dec 15, 2022

Related tags

Deep Learning MixCo-Mixup-Contrast

Overview

How to Reproduce our Results

This repository contains PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation (https://arxiv.org/abs/2010.06300) that is accepted in NeurIPS 2020 Workshop on Self-Supervised Learning.

This is an instruction to reproduce our results, based on the source code we have provided.

Prerequisites

You should download the Tiny-ImageNet dataset. To download the images, go to https://tiny-imagenet.herokuapp.com/ and click 'Download Tiny ImageNet' button. Equivalently, try

wget http://cs231n.stanford.edu/tiny-imagenet-200.zip

Unzip the file and place the folder into [your_data_path].
Also, download the ImageNet train dataset.
For linear classification, you may also need CIFAR10 and CIFAR100 dataset. If you do not have them, give the argument --download when running lincls.py. Then, it will download the dataset before training.

Structure

This repository contains python files that can train the model with mixup-based representaion learning.

pretrain.py pretrains the model in unsupervised manner, and saves the encoder part (without classification layers).

lincls.py loads and freezes the pretrained model, and then train the classifier part on the target dataset.

Experiments

In ./moco/experiments/ and ./simclr/scripts/, there are .sh files which include the commands that can reproduce our experimental results. Open and set the configs.

data_path="[your_data_path]"
exp_name="[experiment_name]"

Run the file. For example, if you want to pretrain the ResNet18 model with Tiny-ImageNet, and then see the linear classification results, run exp_mix_res18_tinyimg.sh.

bash experiments/exp_mix_res18_tinyimg.sh

You can find the pretraining and linear evaluation results in results/results.json file.

Code for our ACL 2021 paper - ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

ConSERT Code for our ACL 2021 paper - ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer Requirements torch==1.6.0

478 Dec 25, 2022

Lightweight mmm - Lightweight (Bayesian) Media Mix Model

Lightweight (Bayesian) Media Mix Model This is not an official Google product. L

342 Jan 3, 2023

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation This repository is the official implementation of CVPR 2021 paper:

9 Nov 14, 2022

Code in conjunction with the publication 'Contrastive Representation Learning for Hand Shape Estimation'

HanCo Dataset & Contrastive Representation Learning for Hand Shape Estimation Code in conjunction with the publication: Contrastive Representation Lea

Computer Vision Group, Albert-Ludwigs-Universität Freiburg

38 Dec 13, 2022

Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training This project hosts the code for implementing the DenseCL algorithm for se

491 Jan 3, 2023

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning This repository contains the code and relevant instructions

5 Aug 19, 2022

Code for Efficient Visual Pretraining with Contrastive Detection

Code for DetCon This repository contains code for the ICCV 2021 paper "Efficient Visual Pretraining with Contrastive Detection" by Olivier J. Hénaff,

56 Nov 13, 2022

This repository is the official implementation of Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning (NeurIPS21).

Core-tuning This repository is the official implementation of ``Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regular

18 Dec 17, 2022

Object-aware Contrastive Learning for Debiased Scene Representation

Object-aware Contrastive Learning Official PyTorch implementation of "Object-aware Contrastive Learning for Debiased Scene Representation" by Sangwoo

43 Dec 14, 2022

Comments

question about the pre-trained models

Thanks for sharing the code of MixCo, which helped us a lot.

As you mentioned in the paper, "ResNet-34 and ResNet-50 encoder are trained for 200 epochs with batch size 128, and ResNet-18 is trained with 100 epochs", I am writing to ask if you can share the pre-trained models? Especially the counterparts in appendix C.2.

Thanks for your work and we will cite this paper in our future work.

opened by lichen14 1

PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

Related tags

Overview

How to Reproduce our Results

Prerequisites

Structure

Experiments

You might also like...

Code for our ACL 2021 paper - ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

Lightweight mmm - Lightweight (Bayesian) Media Mix Model

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Code in conjunction with the publication 'Contrastive Representation Learning for Hand Shape Estimation'

Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

Code for Efficient Visual Pretraining with Contrastive Detection

This repository is the official implementation of Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning (NeurIPS21).

Object-aware Contrastive Learning for Debiased Scene Representation

Comments

question about the pre-trained models

Owner

opcrisis

Codes for the paper Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing

pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Implementation of the Triangle Multiplicative module, used in Alphafold2 as an efficient way to mix rows or columns of a 2d feature map, as a standalone package for Pytorch

PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)

A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

Pytorch Implementation of "Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation"

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)