Official code for the paper "Self-Supervised Prototypical Transfer Learning for Few-Shot Classification"

Related tags

Deep Learning transfer-learning few-shot-learning self-supervised-learning
Overview

Self-Supervised Prototypical Transfer Learning for Few-Shot Classification

This repository contains the reference source code and pre-trained models (ready for evaluation) for our paper Self-Supervised Prototypical Transfer Learning for Few-Shot Classification.

Part of this work has been presented at the ICML 2020 Workshop on Automated Machine Learning.

ProtoTransfer method illustration

Structure

omni-mini/

Contains instructions and all runnable code for ProtoTransfer & UMTRA for our Omniglot and mini-ImageNet experiments

cdfsl-benchmark/

Contains instructions, all runnable code and pre-trained models for ProtoTransfer & UMTRA for our CDFSL benchmark experiments

Setup

For setting up a Python environment to run our experiments, please refer to omni-mini/setup. The dataset setups can be found in omni-mini and cdfsl-benchmark.

Citation

If you find our code useful, please consider citing our work using the bibtex:

@article{medina2020selfsupervised,
    title="{Self-Supervised Prototypical Transfer Learning for Few-Shot Classification}",
    author={Carlos Medina and Arnout Devos and Matthias Grossglauser},
    journal={arXiv preprint arXiv:2006.11325},
    year={2020}
}
Comments
  • Replication on mini-ImageNet

    Replication on mini-ImageNet

    Hi,

    Thanks for uploading great work. It is really exiting to hear your self-supervised learning outperforms MAML on mini-ImageNet!

    I'm trying to replicate your results in the paper. Following the instruction, however, I got slightly degraded accuracy.

    For 1-shot,

    $ python prototransfer/eval.py --dataset miniimagenet --eval_ways 5 --eval_support_shots 1 --eval_query_shots 15 --sup_finetune --ft_freeze_backbone --load_path prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar
Used device: cuda
Supervised data loader for miniimagenet:test.
Loaded checkpoint 'prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar' (epoch 1489)
Evaluating prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar...
Test loss 4.5765 and accuracy 44.41 +- 0.77

    For 5-shot,

    $ python prototransfer/eval.py --dataset miniimagenet --eval_ways 5 --eval_support_shots 5 --eval_que
ry_shots 15 --sup_finetune --ft_freeze_backbone --load_path prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar
Used device: cuda
Supervised data loader for miniimagenet:test.
Loaded checkpoint 'prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar' (epoch 1489)
Evaluating prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs.pth.tar...
Test loss 1.5548 and accuracy 61.38 +- 0.74

    Is it suppose to be similar to the last row in Table 3. (Testing ProtoTune w/ FT)?

    The training stops at epoch 1689, seems a bit early.

    $ python prototransfer/train.py --dataset miniimagenet   --train_support_shots 1   --train_query_shots 3      --no_aug_support                                                                                                                                  Used device: cuda
Save path is: prototransfer/checkpoints/proto_miniimagenet_conv4_euclidean_1supp_3query_50bs
Early stopping with patience 200 epochs
Setting:
Namespace(backbone='conv4', batch_size=50, datapath='../few_data/', dataset='miniimagenet', distance='euclidean', epochs=10000, iterations=100, learn_temperature=False, load_best=False, load_last=False, load_path='', lr=0.001, lr_decay_rate=0.5, lr_decay_step=25000, merge_train_val=False, n_classes=None, n_images=None, no_aug_query=False, no_aug_support=True, num_data_workers_cpu=0, num_data_workers_cuda=8, patience=200, save=True, save_path='', self_sup_loss='proto', train_query_shots=3, train_support_shots=1)

Training ...
Epoch 1690, loss 0.0877, accuracy 0.9701:  17%|██████████▋                                                    | 1689/10000 [2:56:48<14:30:17,  6.28s/epochs]
Early stopping at epoch 1689, because there was no                      improvement for 200 epochs
Epoch 1690, loss 0.0877, accuracy 0.9701:  17%|██████████▋                                                    | 1689/10000 [2:56:48<14:30:03,  6.28s/epochs]
-------------- Best validation loss 0.0756 with accuracy 0.9760

    Any suggestions?

    Best regards, Hwidong Na

    opened by hwidong-na 8
  • Some question about Proto-MAML

    Some question about Proto-MAML

    Good job! I find an interesting phenomenon that the accuracy of ProtoNet's linear-layer form (w=2c, b=-c^2) is significantly lower than that of ProtoNet. However, theoretically the accuracy of the two should be the same. Do you have such a discovery? what is the reason?

    opened by Haoqing-Wang 2
  • some question about the paper

    some question about the paper

    hi ~, it's a great job.

    You introduce self-supervised training before downstream training. So the innovation of the paper is training a good embedding space by self-supervised. The sownstream training is the same as prototypical network. I did some research about self-supervised on resnet50 before. after self-supervised training, I put downstream task's test dataset into model, I visualize the embedding, all classes's embedding is total in a mess, I don't know it is the same in your experiments

    opened by freeman-1995 2
  • the pretrained model tar file is corrupted

    the pretrained model tar file is corrupted

    Hi, I tried to download your trained model in the checkpoint file fold, but can not untar the file, saying the file is corrupted. Can you upload a complete one, please?

    opened by YIYIZH 1
  • Some requests for visualization of the results

    Some requests for visualization of the results

    This is an encouraging work and gives me some inspirations. To better understand some observations in this paper, can you provide t-SNE codes for the visualization of your result?

    Looking forward to your reply!

    opened by CSer-Tang-hao 1
Owner
EPFL INDY
Information and Network Dynamics lab at EPFL
EPFL INDY
GitHub https://arxiv.org/abs/2006.11325
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Oct 6, 2022
Official TensorFlow code for the forthcoming paper

~ Efficient-CapsNet ~ Are you tired of over inflated and overused convolutional neural networks? You're right! It's time for CAPSULES :)

Vittorio Mazzia 203 Jan 8, 2023
Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

ming71 56 Nov 28, 2022
Official code of the paper "ReDet: A Rotation-equivariant Detector for Aerial Object Detection" (CVPR 2021)

ReDet: A Rotation-equivariant Detector for Aerial Object Detection ReDet: A Rotation-equivariant Detector for Aerial Object Detection (CVPR2021), Jiam

csuhan 334 Dec 23, 2022
Official code for the paper: Deep Graph Matching under Quadratic Constraint (CVPR 2021)

QC-DGM This is the official PyTorch implementation and models for our CVPR 2021 paper: Deep Graph Matching under Quadratic Constraint. It also contain

Quankai Gao 55 Nov 14, 2022
Official code for the ICLR 2021 paper Neural ODE Processes

Neural ODE Processes Official code for the paper Neural ODE Processes (ICLR 2021). Abstract Neural Ordinary Differential Equations (NODEs) use a neura

Cristian Bodnar 50 Oct 28, 2022
Official code for the CVPR 2021 paper "How Well Do Self-Supervised Models Transfer?"

How Well Do Self-Supervised Models Transfer? This repository hosts the code for the experiments in the CVPR 2021 paper How Well Do Self-Supervised Mod

Linus Ericsson 157 Dec 16, 2022
This is the official code release for the paper Shape and Material Capture at Home

This is the official code release for the paper Shape and Material Capture at Home. The code enables you to reconstruct a 3D mesh and Cook-Torrance BRDF from one or more images captured with a flashlight or camera with flash.

null 89 Dec 10, 2022
Official source code to CVPR'20 paper, "When2com: Multi-Agent Perception via Communication Graph Grouping"

When2com: Multi-Agent Perception via Communication Graph Grouping This is the PyTorch implementation of our paper: When2com: Multi-Agent Perception vi

null 34 Nov 9, 2022
Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.

Learning Associative Inference Using Fast Weight Memory This repository contains the offical code for the paper Learning Associative Inference Using F

Imanol Schlag 18 Oct 12, 2022
Official PyTorch code for CVPR 2020 paper "Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision"

Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision https://arxiv.org/abs/2003.00393 Abstract Active learning (AL) aims to min

Denis 29 Nov 21, 2022
Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng Internati

Princeton Vision & Learning Lab 115 Jan 4, 2023
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

selfcontact This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] It includes the main function

Lea Müller 68 Dec 6, 2022
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

SMPLify-XMC This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] License Software Copyright Lic

Lea Müller 83 Dec 14, 2022
Official code of paper "PGT: A Progressive Method for Training Models on Long Videos" on CVPR2021

PGT Code for paper PGT: A Progressive Method for Training Models on Long Videos. Install Run pip install -r requirements.txt. Run python setup.py buil

Bo Pang 27 Mar 30, 2022
This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" (PRICAI 2021)

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay This is the official implementation of our paper "Diversity-based Traje

Tianhong Dai 6 Jul 18, 2022
The official code for paper "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling".

R2D2 This is the official code for paper titled "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Mode

Alipay 49 Dec 17, 2022
Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://arxiv.org/abs/2103.06332).

Hurdles to Progress in Long-form Question Answering This repository contains the official scripts and datasets accompanying our NAACL 2021 paper, "Hur

Kalpesh Krishna 41 Nov 8, 2022
Official code for paper "Demystifying Local Vision Transformer: Sparse Connectivity, Weight Sharing, and Dynamic Weight"

Demysitifing Local Vision Transformer, arxiv This is the official PyTorch implementation of our paper. We simply replace local self attention by (dyna

null 138 Dec 28, 2022