Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (CVPR 2021)

Related tags

Deep Learning few-shot-segmentation cvpr2021 pascal-5i prototypical-learning coco-20i
Overview

ASGNet

The code is for the paper "Adaptive Prototype Learning and Allocation for Few-Shot Segmentation" (accepted to CVPR 2021) [arxiv]

Overview

  • data/ includes config files and train/validation list files
  • model/ includes related model and module
  • tool/ includes training and testing scripts
  • util/ includes data processing, seed initialization

Usage

Requirements

python==3.7, torch==1.6, scipy, opencv-python, tensorboardX

Dataset

Prepare related datasets: Pascal-5i (VOC 2012, SBD) and COCO-20i (COCO 2014)

Pre-trained models

  • Pre-trained backbones and models can be found in Google Driver
  • Download backbones and put the pth files under initmodel/ folder

Test and Train

  • Specify the path of datasets and pre-trained models in the data/config file
  • Use the following command 
    sh tool/test.sh|train.sh {data} {model} {split_backbone}

E.g. Test ASGNet with ResNet50 on the split 0 of PASCAL-5i:

sh tool/test.sh pascal asgnet split0_resnet50

Citation

Please consider citing the paper if you find it useful:

@inproceedings{li2021AdaptivePL,
  title={Adaptive Prototype Learning and Allocation for Few-Shot Segmentation},
  author={Gen Li and Varun Jampani and Laura Sevilla-Lara and Deqing Sun and Jonghyun Kim and Joongkyu Kim},
  booktitle={CVPR},
  year={2021}
}

References

The code is based on semseg and PFENet. Thanks for their great work!

Comments
  • The mIoU of split0 is lower than that in the paper

    The mIoU of split0 is lower than that in the paper

    Thank for sharing your code. When I run the training by sh tools/train.sh pascal asgnet split0_resnet50 with your default setting and run the testing by sh tools/test.sh pascal asgnet split0_resnet50 using the checkpoint final.pth, I get the miou of 0.5389 which is much lower than 58.84 in the paper. Do I miss some steps ?

    opened by fangzy97 4
  • how to prepare the datasets?

    how to prepare the datasets?

    Hello, I'm sorry to disturb you, but I am a beginner and I have encountered some problems when preparing the COCO and PASCAL data sets.

    1. After I downloaded the VOC2012 data set, I found that there were only about 2000 pictures, but there were more than 5000 data in ASGNet/data/lists/pascal/voc_sbd_merge_noduplicate.txt. And then, I think maybe I should use VOC2012Aug, but there are more than 10000 pictures in VOC2012Aug.
    2. I downloaded the COCO data train/test/val in the warehouse link and decompressed them, but I did not find the data label. I will be very happy if I can get your help, thank you.
    opened by ZhangIceNight 2
  • The 5-shot evaluation is problematic

    The 5-shot evaluation is problematic

    Hi, thanks for your work.

    I have reproduced the 1-shot results that are close to the ones in your paper.

    However, when I directly test the model trained with 1-shot in the 5-shot evaluation setting, the result is far from yours.

    For example, on fold 0, the 1-shot result is 59.2 and 5-shot I only got 43.73.

    Could you help fix this issue?

    Thankyou.

    opened by happycoding1996 2
  • View the result in the original image

    View the result in the original image

    When I want to view the prediction results of the pictures in the test set, I found that the code I wrote to draw the output results on the original pictures always has bugs and ambiguities. Can I get the code that compares the results in the paper?

    When I am performing image inference, the input is the image to be detected and a picture of the target class and its mask picture, then how can I use a picture to quickly detect different pictures, what I actually want to say is: how It is better to detect unknown pictures, not that I can only use one picture for verification.

    Hope to provide an idea

    opened by Nelebos 2
  • where is coco2012/SegmentationClassAug folder？

    where is coco2012/SegmentationClassAug folder？

    how to get the folder of coco2012/SegmentationClassAug? I download the coco2012 dataset on the internet, but it only has SegmentationClass, without SegmentationClassAug.

    opened by wly-ai-bj 1
  • TypeError: 'int' object is not iterable

    TypeError: 'int' object is not iterable

    Hello, thanks for work! I have some problem when I try to train the model with pascal set. I follow the step show in the github page, and run: sh tool/train.sh pascal asgnet split0_resnet50 but I get the error as follow:

    [2022-02-22 16:10:06,166 INFO train.py line 329 19996] >>>>>>>>>>>>>>>> Start Evaluation >>>>>>>>>>>>>>>> Traceback (most recent call last): File "/opt/conda/envs/dual_learning/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/envs/dual_learning/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/tool/train.py", line 450, in <module> main() File "/home/tool/train.py", line 91, in main main_worker(args.train_gpu, args.ngpus_per_node, args) File "/home/tool/train.py", line 207, in main_worker loss_val, mIoU_val, mAcc_val, allAcc_val, class_miou = validate(val_loader, model, criterion) File "/home/tool/train.py", line 421, in validate allAcc = sum(intersection_meter.sum) / (sum(target_meter.sum) + 1e-10) TypeError: 'int' object is not iterable

    I try to solve this, and I found the "intersection_meter.sum" belongs to "int" type, which leads to the error. As normal, the 'intersection_meter.sum' should be a list or array? Where ma'y I go wrong? Thanks a lot1

    opened by River-mao 2
  • Project request

    Project request

    Hello, first of all, I would like to express my compliments to this network structure you proposed. I think your paper has given me a lot of help, because I am still a beginner and cannot fully implement the project, so I hope you can give me a complete project, I want to see the complete running effect of this project, and hope to get your reply

    opened by Cmnotjx 0
  • Category settings in training and testing

    Category settings in training and testing

    微信图片_20210914154940 Hello thanks for making the code available.

    I have a question. Why are the classes in the configuration file set to 2? These are two specific categories? Or does it have a special meaning?

    opened by asdnjadsn 2
Owner
Gen Li
B.S. Xidian University // M.S. Sungkyunkwan University
Gen Li
GitHub
Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation

Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation The code repository for "Audio-Visual Generalized Few-Shot Learning with

Kaiaicy 3 Jun 27, 2022
Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)'

SCL Introduction Code for 'Self-Guided and Cross-Guided Learning for Few-shot segmentation. (CVPR' 2021)' We evaluated our approach using two baseline

null 34 Oct 8, 2022
Optimal Adaptive Allocation using Deep Reinforcement Learning in a Dose-Response Study

Optimal Adaptive Allocation using Deep Reinforcement Learning in a Dose-Response Study Supplementary Materials for Kentaro Matsuura, Junya Honda, Imad

Kentaro Matsuura 4 Nov 1, 2022
[CVPR 2021] Few-shot 3D Point Cloud Semantic Segmentation

Few-shot 3D Point Cloud Semantic Segmentation Created by Na Zhao from National University of Singapore Introduction This repository contains the PyTor

null 117 Dec 27, 2022
Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learni

null 220 Dec 31, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
Few-NERD: Not Only a Few-shot NER Dataset

Few-NERD: Not Only a Few-shot NER Dataset This is the source code of the ACL-IJCNLP 2021 paper: Few-NERD: A Few-shot Named Entity Recognition Dataset.

THUNLP 319 Dec 30, 2022
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (CVPR 2021) This repo contains the implementation of our state-of-the-art fewshot ob

null 233 Dec 29, 2022
The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

FAPIS The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter Introduction This repo is primari

Khoi Nguyen 8 Dec 11, 2022
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021, official Pytorch implementatio

Microsoft 247 Dec 25, 2022
Official PyTorch Implementation of Hypercorrelation Squeeze for Few-Shot Segmentation, arXiv 2021

Hypercorrelation Squeeze for Few-Shot Segmentation This is the implementation of the paper "Hypercorrelation Squeeze for Few-Shot Segmentation" by Juh

Juhong Min 165 Dec 28, 2022
Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021) This repository is for BAAF-Net introduce

null 90 Dec 29, 2022
Official repository for Few-shot Image Generation via Cross-domain Correspondence (CVPR '21)

Few-shot Image Generation via Cross-domain Correspondence Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, Richard Zh

Utkarsh Ojha 251 Dec 11, 2022
The Pytorch code of "Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification", CVPR 2022 (Oral).

DeepBDC for few-shot learning        Introduction In this repo, we provide the implementation of the following paper: "Joint Distribution Matters: Dee

FeiLong 116 Dec 19, 2022
Portfolio Optimization and Quantitative Strategic Asset Allocation in Python

Riskfolio-Lib Quantitative Strategic Asset Allocation, Easy for Everyone. Description Riskfolio-Lib is a library for making quantitative strategic ass

Riskfolio 1.7k Jan 7, 2023
Code for our method RePRI for Few-Shot Segmentation. Paper at http://arxiv.org/abs/2012.06166

Region Proportion Regularized Inference (RePRI) for Few-Shot Segmentation In this repo, we provide the code for our paper : "Few-Shot Segmentation Wit

Malik Boudiaf 138 Dec 12, 2022
The implementation of PEMP in paper "Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes"

Prior-Enhanced network with Meta-Prototypes (PEMP) This is the PyTorch implementation of PEMP. Overview of PEMP Meta-Prototypes & Adaptive Prototypes

Jianwei ZHANG 8 Oct 14, 2021
Official code for "Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021".

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021. Introduction We proposed a novel model training paradi

Lucas 103 Dec 14, 2022
PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation (TPAMI).

PFENet This is the implementation of our paper PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation that has been accepted to IEE

DV Lab 230 Dec 31, 2022