(ICCV'21) Official PyTorch implementation of Relational Embedding for Few-Shot Classification

Last update: Dec 24, 2022

Overview

Relational Embedding for Few-Shot Classification (ICCV 2021)

Dahyun Kang, Heeseung Kwon, Juhong Min, Minsu Cho

[paper], [project hompage]

We propose to address the problem of few-shot classification by meta-learning “what to observe” and “where to attend” in a relational perspective. Our method leverages relational patterns within and between images via self-correlational representation (SCR) and cross-correlational attention (CCA). Within each image, the SCR module transforms a base feature map into a self-correlation tensor and learns to extract structural patterns from the tensor. Between the images, the CCA module computes cross-correlation between two image representations and learns to produce co-attention between them. (a), (b), and (c) visualize the activation maps of base features, self-correlational representation, and cross-correlational attention, respectively. Our Relational Embedding Network (RENet) combines the two relational modules to learn relational embedding in an end-to-end manner. In experimental evaluation, it achieves consistent improvements over state-of-the-art methods on four widely used few-shot classification benchmarks of miniImageNet, tieredImageNet, CUB-200-2011, and CIFAR-FS.

✔️ Requirements

Ubuntu 16.04
Python 3.7
CUDA 11.0
PyTorch 1.7.1

⚙️ Conda environmnet installation

conda env create --name renet_iccv21 --file environment.yml
conda activate renet_iccv21

📚 Datasets

cd datasets
bash download_miniimagenet.sh
bash download_cub.sh
bash download_cifar_fs.sh
bash download_tieredimagenet.sh

🌳 Authors' checkpoints

cd checkpoints
bash download_checkpoints_renet.sh

The file structure should be as follows:

renet/
├── datasets/
├── model/
├── scripts/
├── checkpoints/
│   ├── cifar_fs/
│   ├── cub/
│   ├── miniimagenet/
│   └── tieredimagenet/
train.py
test.py
README.md
environment.yml

📌 Quick start: testing scripts

To test in the 5-way K-shot setting:

bash scripts/test/{dataset_name}_5wKs.sh

For example, to test ReNet on the miniImagenet dataset in the 5-way 1-shot setting:

bash scripts/test/miniimagenet_5w1s.sh

🔥 Training scripts

To train in the 5-way K-shot setting:

bash scripts/train/{dataset_name}_5wKs.sh

For example, to train ReNet on the CUB dataset in the 5-way 1-shot setting:

bash scripts/train/cub_5w1s.sh

Training & testing a 5-way 1-shot model on the CUB dataset using a TitanRTX 3090 GPU takes 41m 30s.

🎨 Few-shot classification results

Experimental results on few-shot classification datasets with ResNet-12 backbone. We report average results with 2,000 randomly sampled episodes.

datasets	miniImageNet		tieredImageNet
setups	5-way 1-shot	5-way 5-shot	5-way 1-shot	5-way 5-shot
accuracy	67.60	82.58	71.61	85.28

datasets	CUB-200-2011		CIFAR-FS
setups	5-way 1-shot	5-way 5-shot	5-way 1-shot	5-way 5-shot
accuracy	79.49	91.11	74.51	86.60

🔍 Related repos

Our project references the codes in the following repos:

Zhang et al., DeepEMD.
Ye et al., FEAT
Wang et al., Non-local neural networks
Ramachandran et al., Stand-alone self-attention
Huang et al., DCCNet
Yang et al., VCN

💌 Acknowledgement

We adopted the main code bases from DeepEMD, and we really appreciate it 😃 . We also sincerely thank all the ICCV reviewers, especially R#2, for valuable suggestions.

📜 Citing RENet

If you find our code or paper useful to your research work, please consider citing our work using the following bibtex:

@inproceedings{kang2021renet,
    author   = {Kang, Dahyun and Kwon, Heeseung and Min, Juhong and Cho, Minsu},
    title    = {Relational Embedding for Few-Shot Classification},
    booktitle= {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year     = {2021}
}

Comments

Could you please provide the code of RENet applied to Grad-cam?

Hi~ The support set and query set visualized by Grad-cam is really beautiful in this article. Could you please provide the code of RENet applied to Grad-cam? I'd like to try that!

opened by woodszp 6
Paper results

Very interesting work!

Hi, I am very interested in your work. I would like to ask how to replicate the results in table 3 in the main paper? More specifically, I want to study the effect of the two modules by switching on/off them as in table 3 .

opened by kltrock 2
Testing Output Network

Hi, I'd like to give an image input from testset, in order to visualize the result (the corrisponding label).
How can I do this? Is there a command? Thanks

opened by AleLdf 2
Inductive or transductive？

In the n-way k-shot( k>1) setting, the attended features of the support set are computed by summing k attended features, which are influenced by the query set. I wonder whether it is an inductive or transductive setting in few-shot learning.

opened by Fancy-sf 1
Conv3d or Conv4d in SCR

Hi, thanks for releasing the organized code.

I find that in the code SCR is implemented with Conv3d, while in the paper it is Conv4d. Does this matter?

opened by csyanbin 1

(ICCV'21) Official PyTorch implementation of Relational Embedding for Few-Shot Classification

Related tags

Overview

Relational Embedding for Few-Shot Classification (ICCV 2021)

Dahyun Kang, Heeseung Kwon, Juhong Min, Minsu Cho

[paper], [project hompage]

✔️ Requirements

⚙️ Conda environmnet installation

📚 Datasets

🌳 Authors' checkpoints

📌 Quick start: testing scripts

🔥 Training scripts

🎨 Few-shot classification results

🔍 Related repos

💌 Acknowledgement

📜 Citing RENet

Comments

Could you please provide the code of RENet applied to Grad-cam?

Paper results

Testing Output Network

Inductive or transductive？

Conv3d or Conv4d in SCR

Owner

Dahyun Kang

An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Code for the paper "Query Embedding on Hyper-relational Knowledge Graphs"

Few-NERD: Not Only a Few-shot NER Dataset

Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).

An original implementation of "Noisy Channel Language Model Prompting for Few-Shot Text Classification"

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

[ICCV 2021] Official PyTorch implementation for Deep Relational Metric Learning.

Official Pytorch Implementation of Relational Self-Attention: What's Missing in Attention for Video Understanding

Ready-to-use code and tutorial notebooks to boost your way into few-shot image classification.

Library of various Few-Shot Learning frameworks for text classification

Spatial Contrastive Learning for Few-Shot Classification (SCL)

TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

vit for few-shot classification

Official Implementation of Few-shot Visual Relationship Co-localization

The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

Official Pytorch Implementation of: "Semantic Diversity Learning for Zero-Shot Multi-label Classification"(2021) paper

Pytorch Implementation of Adversarial Deep Network Embedding for Cross-Network Node Classification