[ICCV 2021 Oral] Mining Latent Classes for Few-shot Segmentation

Related tags

Deep Learning semantic-segmentation few-shot-learning few-shot-segmentation iccv2021
Overview

Mining Latent Classes for Few-shot Segmentation

Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, Yang Gao.

This codebase contains baseline of our paper Mining Latent Classes for Few-shot Segmentation, ICCV 2021 Oral.

Several key modifications to the simple yet effective metric learning framework:

  • Remove the final residual stage in ResNet for stronger generalization
  • Remove the final ReLU for feature matching
  • Freeze all the BatchNorms from ImageNet pretrained model

Environment

This codebase was tested with the following environment configurations.

  • Ubuntu 18.04
  • CUDA 11.2
  • Python 3.7.4
  • PyTorch 1.6.0
  • Pillow, numpy, torchvision, tqdm
  • Two NVIDIA V100 GPUs

Getting Started

Data Preparation

Pretrained model: ResNet-50 | ResNet-101

Dataset: Pascal JPEGImages | SegmentationClass | ImageSets

File Organization

├── ./pretrained
    ├── resnet50.pth
    └── resnet101.pth
    
├── [Your Pascal Path]
    ├── JPEGImages
    │   ├── 2007_000032.jpg
    │   └── ...
    │
    ├── SegmentationClass
    │   ├── 2007_000032.png
    │   └── ...
    │
    └── ImageSets
        ├── train.txt
        └── val.txt

Run the Code

CUDA_VISIBLE_DEVICES=0,1 python -W ignore main.py \
  --dataset pascal --data-root [Your Pascal Path] \
  --backbone resnet50 --fold 0 --shot 1

You may change the backbone from resnet50 to resnet101, change the fold from 0 to 1/2/3, or change the shot from 1 to 5 for other settings.

Performance and Trained Models

Here we report the performance of our modified baseline on Pascal. You can click on the numbers to download corresponding trained models.

The training time is measured on two V100 GPUs. Compared with other works, our method is efficient to train.

Setting Backbone Training time / fold Fold 0 Fold 1 Fold 2 Fold 3 Mean
1-shot ResNet-50 40 minutes 54.9 66.5 61.7 48.3 57.9
1-shot ResNet-101 1.1 hours 57.2 68.5 61.3 53.3 60.1
5-shot ResNet-50 2.3 hours 61.6 70.3 70.5 56.4 64.7
5-shot ResNet-101 3.5 hours 64.2 74.0 71.5 61.3 67.8

Acknowledgement

We thank PANet, PPNet, PFENet and other FSS works for their great contributions.

Citation

If you find this project useful for your research, please consider citing:

@inproceedings{yang2021mining,
  title={Mining Latent Classes for Few-shot Segmentation},
  author={Yang, Lihe and Zhuo, Wei and Qi, Lei and Shi, Yinghuan and Gao, Yang},
  booktitle={ICCV},
  year={2021}
}
Comments
  • Offline Annotation and Mining Embedding

    Offline Annotation and Mining Embedding

    Hello, thank you for your great contribution! I have a question: Can you provide the code of Offline Annotation and Mining Embedding? It seems like the repository doesn't include them.

    opened by CARRLIANSS 2
  • About Trained Models

    About Trained Models

    Hello, thank you for your great contribution! I have a question: trained model is pretrained backbone? It seems like each trained model which you provided is not the best model.

    opened by CARRLIANSS 2
  • Data replacement

    Data replacement

    Hello, I realized the effect of the paper. I changed the experimental data into my own data. Classification category: 1, 2, 3. The label image is as follows QQ截图20220513153034 For my above classification, how to change fewshow Py code, the screenshot is as follows 21 Thank you for your guidance

    opened by AI-student-wl 1
  • About pretrained embedding network on offline annotation

    About pretrained embedding network on offline annotation

    I have a question in the offline annotation part. It is mentioned that it used the pretrained embedding network, does the pretrained network mean the imagenet pretrained weight? If not, may I know how did you learn the model?

    opened by Jyerim 1
  • Prototype Rectification

    Prototype Rectification

    Does this code contains any prototype rectification process mentioned in the paper ? I do not observe such moving average operation during episodic training. I think you share training without rectification. Is it true ?

    opened by alpoler 1
Owner
Lihe Yang
Master student at Nanjing University, Computer Vision
Lihe Yang
GitHub https://arxiv.org/abs/2103.15402
The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).

DeepBDC for few-shot learning        Introduction In this repo, we provide the implementation of the following paper: "Joint Distribution Matters: Dee

FeiLong 116 Dec 19, 2022
Few-NERD: Not Only a Few-shot NER Dataset

Few-NERD: Not Only a Few-shot NER Dataset This is the source code of the ACL-IJCNLP 2021 paper: Few-NERD: A Few-shot Named Entity Recognition Dataset.

THUNLP 319 Dec 30, 2022
Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learni

null 220 Dec 31, 2022
Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Hypercorrelation Squeeze for Few-Shot Segmentation This is the implementation of the paper "Hypercorrelation Squeeze for Few-Shot Segmentation" by Juh

Juhong Min 165 Dec 28, 2022
[CVPR 2021] Few-shot 3D Point Cloud Semantic Segmentation

Few-shot 3D Point Cloud Semantic Segmentation Created by Na Zhao from National University of Singapore Introduction This repository contains the PyTor

null 117 Dec 27, 2022
Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (CVPR 2021)

ASGNet The code is for the paper "Adaptive Prototype Learning and Allocation for Few-Shot Segmentation" (accepted to CVPR 2021) [arxiv] Overview data/

Gen Li 91 Dec 23, 2022
Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)'

SCL Introduction Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)' We evaluated our approach using two baseline

null 34 Oct 8, 2022
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
(ICCV'21) Official PyTorch implementation of Relational Embedding for Few-Shot Classification

Relational Embedding for Few-Shot Classification (ICCV 2021) Dahyun Kang, Heeseung Kwon, Juhong Min, Minsu Cho [paper], [project hompage] We propose t

Dahyun Kang 82 Dec 24, 2022
An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation This is an official implementation of the paper "Exploiting a Joint

CV Lab @ Yonsei University 35 Oct 26, 2022
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

Christos Tzelepis 100 Dec 6, 2022
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

Malik Boudiaf 138 Dec 12, 2022
The implementation of PEMP in paper "Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes"

Prior-Enhanced network with Meta-Prototypes (PEMP) This is the PyTorch implementation of PEMP. Overview of PEMP Meta-Prototypes & Adaptive Prototypes

Jianwei ZHANG 8 Oct 14, 2021
Official code for "Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021".

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021. Introduction We proposed a novel model training paradi

Lucas 103 Dec 14, 2022
PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation (TPAMI).

PFENet This is the implementation of our paper PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation that has been accepted to IEE

DV Lab 230 Dec 31, 2022
The code is for the paper "A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation"

SD-AANet The code is for the paper "A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation" [arxiv] Overview confi

cv516Buaa 9 Nov 7, 2022
PyTorch implementation of: Michieli U. and Zanuttigh P., "Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations", CVPR 2021.

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations This is the official PyTorch implementation

Multimedia Technology and Telecommunication Lab 42 Nov 9, 2022
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (CVPR 2021) This repo contains the implementation of our state-of-the-art fewshot ob

null 233 Dec 29, 2022
Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

MetaAdaptRank This repository provides the implementation of meta-learning to reweight synthetic weak supervision data described in the paper Few-Shot

THUNLP 5 Jun 16, 2022