Parametric Contrastive Learning (ICCV2021)

Related tags

Deep Learning pytorch supervised-learning imagenet image-classification class-imbalance imbalanced-data imbalanced-learning contrastive-learning supervised-contrastive-learning long-tailed-recognition iccv2021 parametric-contrastive-learning
Overview

Parametric-Contrastive-Learning

This repository contains the implementation code for ICCV2021 paper:
Parametric Contrastive Learning (https://arxiv.org/abs/2107.12028)

PWC

PWC

PWC

If you find this code or idea useful, please consider citing our work:

@misc{cui2021parametric,
      title={Parametric Contrastive Learning}, 
      author={Jiequan Cui and Zhisheng Zhong and Shu Liu and Bei Yu and Jiaya Jia},
      year={2021},
      eprint={2107.12028},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Overview

In this paper, we propose Parametric Contrastive Learning (PaCo) to tackle long-tailed recognition. Based on theoretical analysis, we observe supervised contrastive loss tends to bias on high-frequency classes and thus increases the difficulty of imbalance learning. We introduce a set of parametric class-wise learnable centers to rebalance from an optimization perspective. Further, we analyze our PaCo loss under a balanced setting. Our analysis demonstrates that PaCo can adaptively enhance the intensity of pushing samples of the same class close as more samples are pulled together with their corresponding centers and benefit hard example learning. Experiments on long-tailed CIFAR, ImageNet, Places, and iNaturalist 2018 manifest the new state-of-the-art for long-tailed recognition. On full ImageNet, models trained with PaCo loss surpass supervised contrastive learning across various ResNet backbones.

Results and Pretrained models

Full ImageNet (Balanced setting)

Method Model Top-1 Acc(%) link log
PaCo ResNet-50 79.3 download download
PaCo ResNet-101 80.9 download download
PaCo ResNet-200 81.8 download download

ImageNet-LT (Imbalance setting)

Method Model Top-1 Acc(%) link log
PaCo ResNet-50 57.0 download download
PaCo ResNeXt-50 58.2 download download
PaCo ResNeXt-101 60.0 download download

iNaturalist 2018 (Imbalanced setting)

Method Model Top-1 Acc(%) link log
PaCo ResNet-50 73.2 download download
PaCo ResNet-152 75.2 download download

Places-LT (Imbalanced setting)

Method Model Top-1 Acc(%) link log
PaCo ResNet-152 41.2 download download

Get Started

For full ImageNet, ImageNet-LT, iNaturalist 2018, Places-LT training and evaluation. Note that PyTorch>=1.6. All experiments are conducted on 4 GPUs. If you have more GPU resources, please make sure that the learning rate should be linearly scaled and 32 images per gpu is recommented.

cd Full-ImageNet
bash sh/train_resnet50.sh
bash sh/eval_resnet50.sh

cd LT
bash sh/ImageNetLT_train_R50.sh
bash sh/ImageNetLT_eval_R50.sh
bash sh/PlacesLT_train_R152.sh
bash sh/PlacesLT_eval_R152.sh

Contact

If you have any questions, feel free to contact us through email ([email protected]) or Github issues. Enjoy!

Comments
  • Question about CIFAR-LT experiment

    Question about CIFAR-LT experiment

    Hi! thank you for your interesting work!

    I have a question on hyperparameters in CIFAR-LT dataset experiments

    The paper explains alpha and moco temperature, but there is no explanation for the rest of the hyper-parameters(beta, gamma, augmentation stradegy). Can you please provide it?

    opened by caisarl76 6
  • Questions about experiments

    Questions about experiments

    Thanks for your ICCV work. However, I find you directly use the test set of ImageNet-LT to store the best models, which may lead to overfitting in practice and seems to be unfair to other compared methods. Could you please provide the results on ImageNet-LT using the validation set to store models? It would be easier for us to compare with PaCo in our work. Thanks very much.

    opened by Hongbin98 6
  • Questions about tau-norm with randaugment

    Questions about tau-norm with randaugment

    Thanks for your exciting work! Tau-norm with randaugment performs so well as shown in Table 3 and Table 5. I wonder about its implementation, just use augmentation_randncls as train_transform in training stage-1?

    opened by adf1178 5
  • Questions about NormedLinear_Classifier

    Questions about NormedLinear_Classifier

    Thanks for your great work. I am trying to use NormedLinear_Classifier since I may modify it later. But, it is not good now. Could you please tell me the hyperparameters when your training, such as lr, supt in loss?

    opened by JingLiJJ 4
  • The meaning of beta and gamma in the code

    The meaning of beta and gamma in the code

    parser.add_argument('--beta', default=1.0, type=float, help='supervise loss weight') parser.add_argument('--gamma', default=1.0, type=float, help='paco loss') I found these two parameters in the code, and they are used in losses.py, but I can't understand the using of them, are they mentioned in the paper? If you can explain them to me, I will very appreciate on you, thank you!

    opened by madoka109 4
  • The checkpoint on iNat2018

    The checkpoint on iNat2018

    Thank you guys for your impressive work and releasing the code. I just wonder when will the checkpoint on iNat2018 be released? I'm looking forward to that. Thanks a lot!

    opened by Duconnor 3
  • FileNotFoundError: [Errno 2] No such file or directory: 'pretrained_model/moco_ckpt.best.pth.tar' raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

    FileNotFoundError: [Errno 2] No such file or directory: 'pretrained_model/moco_ckpt.best.pth.tar' raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

    Hi, Thank you so much for your work. I am facing this issue while using this repo. I don't know where to upload this file because this folder is not there.

    opened by noreenanwar 1
  • Different resnet backbones

    Different resnet backbones

    Hi thanks for the great work. I notice that you have used different renet backbones for cifar training (load from resnet_cifar) and imagenet training (load from resnet_imagenet). Is there a reason why you use different backbones?

    opened by thomascong121 1
  • Question about inference function in moco/builder.py

    Question about inference function in moco/builder.py

    Hello, I'm impressed with your work and thank you for sharing the codes.

    I have a question abut inference function in LT/moco/builder.py ! In the code below, you are not using q but using self.feat_after_avg_q as input for linear. Then, it doesn't seem to need the first two lines in the code. Or, should self.feat_after_avg_q be changed to q? Can you check this? Thank you in advance! :)

    _def _inference(self, image): q = self.encoder_q(image) q = nn.functional.normalize(q, dim=1) encoder_q_logits = self.linear(self.feat_after_avg_q)

        return encoder_q_logits_
    opened by pseulki 1
  • Question about supcon framework

    Question about supcon framework

    Hi, thanks for the great work, I have a question about the selection of the framework. Since you use the supervised contrastive loss, why not use the framework from supcon and use MoCo framework instead?

    opened by thomascong121 1
  • Questions about center learning

    Questions about center learning

    Hi, thanks for your exciting work.
    I have a question about the centre learning that I did not find any function in the paper, and I did not find any comment of centres on the code. Could you give any clue about how could we the parametric centres?

    I am looking forward to your reply.

    opened by zuglerQ 1
Owner
DV Lab
Deep Vision Lab
DV Lab
GitHub https://arxiv.org/abs/2107.12028
A non-linear, non-parametric Machine Learning method capable of modeling complex datasets

Fast Symbolic Regression Symbolic Regression is a non-linear, non-parametric Machine Learning method capable of modeling complex data sets. fastsr aim

VAMSHI CHOWDARY 3 Jun 22, 2022
SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING" i

Saeed Lotfi 28 Dec 12, 2022
A parametric soroban written with CADQuery.

A parametric soroban written in CADQuery The purpose of this project is to demonstrate how "code CAD" can be intuitive to learn. See soroban.py for a

Lee 4 Aug 13, 2022
The personal repository of the work: *DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer*.

DanceNet3D The personal repository of the work: DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer. Dataset and Results Pleas

南嘉Nanga 36 Dec 21, 2022
Official implementation of NPMs: Neural Parametric Models for 3D Deformable Shapes - ICCV 2021

NPMs: Neural Parametric Models Project Page | Paper | ArXiv | Video NPMs: Neural Parametric Models for 3D Deformable Shapes Pablo Palafox, Aljaz Bozic

PabloPalafox 109 Nov 22, 2022
The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

DAG Amendment for Inverse Control of Parametric Shapes This repository is the official Blender implementation of the paper "DAG Amendment for Inverse

Elie Michel 157 Dec 26, 2022
Adversarial Framework for (non-) Parametric Image Stylisation Mosaics

Fully Adversarial Mosaics (FAMOS) Pytorch implementation of the paper "Copy the Old or Paint Anew? An Adversarial Framework for (non-) Parametric Imag

Zalando Research 120 Dec 24, 2022
This python-based package offers a way of creating a parametric OpenMC plasma source from plasma parameters.

openmc-plasma-source This python-based package offers a way of creating a parametric OpenMC plasma source from plasma parameters. The OpenMC sources a

Fusion Energy 10 Oct 18, 2022
JumpDiff: Non-parametric estimator for Jump-diffusion processes for Python

jumpdiff jumpdiff is a python library with non-parametric Nadaraya─Watson estimators to extract the parameters of jump-diffusion processes. With jumpd

Rydin 28 Dec 10, 2022
This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

HeadNeRF: A Real-time NeRF-based Parametric Head Model This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametr

null 294 Jan 1, 2023
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
[ICCV2021] Learning to Track Objects from Unlabeled Videos

Unsupervised Single Object Tracking (USOT) ?? Learning to Track Objects from Unlabeled Videos Jilai Zheng, Chao Ma, Houwen Peng and Xiaokang Yang 2021

null 53 Dec 28, 2022
Semi-Supervised Learning, Object Detection, ICCV2021

End-to-End Semi-Supervised Object Detection with Soft Teacher By Mengde Xu*, Zheng Zhang*, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai,

Microsoft 789 Dec 27, 2022
Dynamic Attentive Graph Learning for Image Restoration, ICCV2021 [PyTorch Code]

Dynamic Attentive Graph Learning for Image Restoration This repository is for GATIR introduced in the following paper: Chong Mou, Jian Zhang, Zhuoyuan

Jian Zhang 84 Dec 9, 2022
Official Repo for ICCV2021 Paper: Learning to Regress Bodies from Images using Differentiable Semantic Rendering

[ICCV2021] Learning to Regress Bodies from Images using Differentiable Semantic Rendering Getting Started DSR has been implemented and tested on Ubunt

Sai Kumar Dwivedi 83 Nov 27, 2022
Official PyTorch Implementation of Rank & Sort Loss [ICCV2021]

Rank & Sort Loss for Object Detection and Instance Segmentation The official implementation of Rank & Sort Loss. Our implementation is based on mmdete

Kemal Oksuz 229 Dec 20, 2022
source code of “Visual Saliency Transformer” (ICCV2021)

Visual Saliency Transformer (VST) source code for our ICCV 2021 paper “Visual Saliency Transformer” by Nian Liu, Ni Zhang, Kaiyuan Wan, Junwei Han, an

null 89 Dec 21, 2022
HiFT: Hierarchical Feature Transformer for Aerial Tracking (ICCV2021)

HiFT: Hierarchical Feature Transformer for Aerial Tracking Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, and Yiming Li Our paper is Accepted by ICCV 2

Intelligent Vision for Robotics in Complex Environment 55 Nov 23, 2022
Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

HU Zeyu 82 Dec 27, 2022