This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

Related tags

Deep Learning JigsawClustering
Overview

JigsawClustering

Jigsaw Clustering for Unsupervised Visual Representation Learning

Pengguang Chen, Shu Liu, Jiaya Jia

Introduction

This project provides an implementation for the CVPR 2021 paper "Jigsaw Clustering for Unsupervised Visual Representation Learning"

Installation

Environment

We verify our code on

  • 4x2080Ti GPUs
  • CUDA 10.1
  • python 3.7
  • torch 1.6.0
  • torchvision 0.7.0

Other similar envirouments should also work properly.

Install

We use the SyncBN from apex, please install apex refer to https://github.com/NVIDIA/apex (SyncBN from pytorch should also work properly, we will verify it later.)

We use detectron2 for the training of detection tasks. If you are willing to finetune our pretrained model on the detection task, please install detectron2 refer to https://github.com/facebookresearch/detectron2

git clone https://github.com/Jia-Research-Lab/JigsawClustering.git
cd JigsawClustering/
pip install diffdist

Dataset

Please put the data under ./datasets. The directory looks like:

datasets
│
│───ImageNet/
│   │───class1/
│   │───class2/
│   │   ...
│   └───class1000/
│   
│───coco/
│   │───annotations/
│   │───train2017/
│   └───val2017/
│
│───VOC2012/
│   
└───VOC2007/

Results and pretrained model

The pretrained model is available at here.

Task Dataset Results
Linear Evaluation ImageNet 66.4
Semi-Supervised 1% ImageNet 40.7
Semi-Supervised 10% ImageNet 63.0
Detection COCO 39.3

Training

Pre-training on ImageNet

python main.py --dist-url 'tcp://localhost:10107' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --lr 0.03 --batch-size 256 --epoch 200 \
    --save-dir outputs/jigclu_pretrain/ \
    --resume outputs/jigclu_pretrain/model_best.pth.tar \
    --loss-t 0.3 \
    --cross-ratio 0.3 \
    datasets/ImageNet/

Linear evaluation on ImageNet

python main_lincls.py --dist-url 'tcp://localhost:10007' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --lr 10.0 --batch-size 256 \
    --prefix module.encoder. \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_linear/ \
    datasets/ImageNet/

Semi-Supervised finetune on ImageNet

10% label

python main_semi.py --dist-url 'tcp://localhost:10102' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --batch-size 256 \
    --wd 0.0 --lr 0.01 --lr-last-layer 0.2 \
    --syncbn \
    --prefix module.encoder. \
    --labels-perc 10 \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_semi_10p/ \
    datasets/ImageNet/

1% label

python main_semi.py --dist-url 'tcp://localhost:10101' --multiprocessing-distributed --world-size 1 --rank 0 \
    -a resnet50 \
    --batch-size 256 \
    --wd 0.0 --lr 0.02 --lr-last-layer 5.0 \
    --syncbn \
    --prefix module.encoder. \
    --labels-perc 1 \
    --pretrained outputs/jigclu_pretrain/model_best.pth.tar \
    --save-dir outputs/jigclu_semi_1p/ \
    datasets/ImageNet/

Transfer to COCO detection

Please convert the pretrained weight first

python detection/convert.py

Then start training using

python detection/train_net.py --config-file detection/configs/R50-JigClu.yaml --num-gpus 4

VOC detection

python detection/train_net.py --config-file detection/configs/voc-R50-JigClu.yaml --num-gpus 4

Citation

Please consider citing JigsawClustering in your publications if it helps your research.

@inproceedings{chen2021jigclu,
    title={Jigsaw Clustering for Unsupervised Visual Representation Learning},
    author={Pengguang Chen, Shu Liu, and Jiaya Jia},
    booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2021},
}
Comments
  • Some question about trainning

    Some question about trainning

    Hi~Thanks for your excellent work! I have a machine with 2 1080Ti,and I want to train your model on CIFAR10 with resnet18.

    I use the parmeters like this ,but it seems don't work. 1632405015(1)

    The program is stuck in this situation.

    1632405115(1)

    opened by zbw0329 10
  • Some details about the training

    Some details about the training

    Hi, I have recently read your paper and find it very interesting. There are still some confusions about the experiments.

    The experiments require 4 2080ti for training. Does it mean we must have 4 2080ti on one single machine? What if I have 4 2080ti on different machines? Is there any suggestion for this situation? BTW, how long does it take when you train on ImageNet1k?

    Much appreciation for your reply.

    Best wishes!

    opened by Hanzy1996 3
  • Some questions about the results of ImageNet100

    Some questions about the results of ImageNet100

    Thank you for your wonderful work, I want to do some more works based on your code. But I meet some questions about the results. I use the JigsawClustering and the dataset ImageNet100 to train the model. I only changed one line in the model to fit this dataset(I added model.fc = nn.Linear(2048, 100) in line 162 of main_lincls.py). However, despite using 4 GPUs, and did not change the configuration file. I only got an accuracy of 79.24. There is still a certain gap between this and the 80.9 reported in the paper. How can I achieve the accuracy reported in the paper now? Once again, thank you for your excellent work and code. I am looking forward to your reply.

    opened by WilyZhao8 1
  • Results of Faster-RCNN R50-FPN with model pretrained on ImageNet with standard cross-entropy loss

    Results of Faster-RCNN R50-FPN with model pretrained on ImageNet with standard cross-entropy loss

    Hi, thanks for your work! In Objection Detection, do you apply ResNet-50 model pretrained on ImageNet with standard cross-entropy loss to Faster-RCNN R50-FPN?

    opened by fzfs 1
  • Training the model on a single GPU

    Training the model on a single GPU

    Hi! I'm aware that the question has been asked previously, but could you guide how to modify jigclu to remove the distributeddataparallel depedency?

    Thanks!

    opened by shuvam-creditmate 2
  • It seems that the model has not learned anything，What should I do？

    It seems that the model has not learned anything，What should I do？

    Thanks for your excellent work! I change the dataloader to use JigClu in CIFAR-10,and train the model on it by 1000epoch. But the prediction of my model is all the same. It seem that model always cluster into the same cluster

    opened by zbw0329 10
Releases(1.0)
Owner
DV Lab
Deep Vision Lab
DV Lab
GitHub
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning, CVPR 2021

Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning By Zhenda Xie*, Yutong Lin*, Zheng Zhang, Yue Ca

Zhenda Xie 293 Dec 20, 2022
Jigsaw Rate Severity of Toxic Comments

Jigsaw Rate Severity of Toxic Comments

Guanshuo Xu 66 Nov 30, 2022
[CVPR 2021] Unsupervised Degradation Representation Learning for Blind Super-Resolution

DASR Pytorch implementation of "Unsupervised Degradation Representation Learning for Blind Super-Resolution", CVPR 2021 [arXiv] Overview Requirements

Longguang Wang 318 Dec 24, 2022
Code for Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021)

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021) Hang Zhou, Yasheng Sun, Wayne Wu, Chen Cha

Hang_Zhou 628 Dec 28, 2022
git git《Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking》(CVPR 2021) GitHub:git2] 《Masksembles for Uncertainty Estimation》(CVPR 2021) GitHub:git3]

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li Accepted by CVPR

NingWang 236 Dec 22, 2022
[CVPR'21 Oral] Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning [CVPR'21, Oral] By Zhicheng Huang*, Zhaoyang Zeng*, Yupan H

Multimedia Research 196 Dec 13, 2022
[CVPR 2022 Oral] Rethinking Minimal Sufficient Representation in Contrastive Learning

Rethinking Minimal Sufficient Representation in Contrastive Learning PyTorch implementation of Rethinking Minimal Sufficient Representation in Contras

null 36 Nov 23, 2022
[CVPR 2022 Oral] Crafting Better Contrastive Views for Siamese Representation Learning

Crafting Better Contrastive Views for Siamese Representation Learning (CVPR 2022 Oral) 2022-03-29: The paper was selected as a CVPR 2022 Oral paper! 2

null 249 Dec 28, 2022
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning This is the official PyTorch implementation for UniMoCo pape

dddzg 49 Jan 2, 2023
Graph Regularized Residual Subspace Clustering Network for hyperspectral image clustering

Graph Regularized Residual Subspace Clustering Network for hyperspectral image clustering

Yaoming Cai 5 Jul 18, 2022
Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods

ADGC: Awesome Deep Graph Clustering ADGC is a collection of state-of-the-art (SOTA), novel deep graph clustering methods (papers, codes and datasets).

yueliu1999 297 Dec 27, 2022
Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

Facebook Research 182 Dec 30, 2022
(CVPR 2022 Oral) Official implementation for "Surface Representation for Point Clouds"

RepSurf - Surface Representation for Point Clouds [CVPR 2022 Oral] By Haoxi Ran* , Jun Liu, Chengjie Wang ( * : corresponding contact) The pytorch off

Haoxi Ran 264 Dec 23, 2022
code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? Code for paper: Does Unsupervised Architecture Representation

null 39 Dec 17, 2022
[AAAI2021] The source code for our paper 《Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion》.

DSM The source code for paper Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion Project Website; Datasets li

Jinpeng Wang 114 Oct 16, 2022
the code of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021)

RMA-Net This repo is the implementation of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021). Paper

Wanquan Feng 205 Nov 9, 2022
PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

How to Reproduce our Results This repository contains PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Represen

opcrisis 46 Dec 15, 2022
《Improving Unsupervised Image Clustering With Robust Learning》(2020)

Improving Unsupervised Image Clustering With Robust Learning This repo is the PyTorch codes for "Improving Unsupervised Image Clustering With Robust L

Sungwon Park 129 Dec 27, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022