This is an official implementation for "ResT: An Efficient Transformer for Visual Recognition".

Related tags

Deep Learning ResT
Overview

ResT

By Qing-Long Zhang and Yu-Bin Yang

[State Key Laboratory for Novel Software Technology at Nanjing University]

This repo is the official implementation of "ResT: An Efficient Transformer for Visual Recognition". It currently includes code and models for the following tasks:

Image Classification: Included in this repo. See get_started.md for a quick start.

Object Detection and Instance Segmentation: Based on detectron2, coming soon.

ResT is initially described in arxiv, which capably serves as a general-purpose backbone for computer vision. It can tackle input images with arbitrary size. Besides, ResT compressed the memory of standard MSA and model the interaction between multi-heads while keeping the diversity ability.

Main Results on ImageNet with Pretrained Models

ImageNet-1K Pretrained Models

name resolution acc@1 acc@5 #params FLOPs FPS 1K model
ResT-Lite 224x224 77.2 93.7 10.5M 1.4G 1246 baidu
ResT-Small 224x224 79.6 94.9 13.7M 1.9G 1043 baidu
ResT-Base 224x224 81.6 95.7 30.3M 4.3G 673 baidu
ResT-Large 224x224 83.6 96.3 51.6M 7.9G 429 baidu

Note: access code for baidu is rest.

Citing ResT

@article{zhql2021ResT,
  title={ResT: An Efficient Transformer for Visual Recognition},
  author={Zhang, Qinglong and Yang, Yubin},
  journal={arXiv preprint arXiv:2105.13677v2},
  year={2021}
}
Comments
  • Is there any quantitative analysis of the experimental results of MSA and EMSA?

    Is there any quantitative analysis of the experimental results of MSA and EMSA?

    I think it is a necessary ablation study to make a quantitative comparison of performance and efficiency of the two modules. But it's not in the paper

    opened by superPangpang 3
  • Performance in MVS

    Performance in MVS

    Hi~ Thanks for your excellent work! I see this words in your code: x = self.avg_pool(x).flatten(1) # if use in MVS, should abandon this part x = self.head(x) # if use in MVS, should abandon this part It seems you have tried Rest in MVS task, I'm wandering what the perfomance of Rest in MVS, and what experiment you have tried. Looking forward to your reply! Thanks!

    opened by DingYikang 1
  • About finetune

    About finetune

    Hello,when I use your ResT_base weight to my fine-frained image classification task, I use the "--finetune",but i got a mistake . in 264 line(main.py) : **UnboundLocalError: local variable 'model_without_ddp' referenced before assignment.**how can i fix it?

    by the way, in line 431 in main.py: n_parameters is not define,can you give the definition？

    opened by Christine620 1
  • Each head in MSA only responsible for a subset of the input tokens ？

    Each head in MSA only responsible for a subset of the input tokens ？

    maybe "Each head in MSA only responsible for a subset of channels of the input tokens". According to the Transformer, each head is not responsible for part of input tokens, but part of channels of one input token.

    opened by valencebond 1
  • How to train resT, not resT-v2

    How to train resT, not resT-v2

    Hello. I want to train resT on my own dataset. Therefore, I tried to write the model name 'rest_base' on args.model since this name exists in rest.py file. However the error 'unrecognized arguments: -model rest_base' occured.

    So which commands can I enter into args.model in order to train resT v1?

    opened by Jihyepark98 1
  • How to use ResT as my own backbone network

    How to use ResT as my own backbone network

    Hi, thanks for your great work. Now I want to use your Rest as my backbone network for visual target tracking. How do I implement this operation? Can I directly download your pre-trained model for loading?

    opened by hongsheng-Z 2
  • Is there any operation similar to ‘padding mask’?

    Is there any operation similar to ‘padding mask’?

    Hi, thanks for your great work, and Is there any operation similar to 'padding mask' like as DERT to indicate where is the image and where is padding.

    opened by hongsheng-Z 0
  • KeyError: 'Non-existent config key: MODEL.REST'

    KeyError: 'Non-existent config key: MODEL.REST'

    你好，我安装了detectron2，直接在d2文件夹下，运行./train_net.py --num-gpus 1
    --config-file ./configs/COCO-InstanceSegmentation/mask_rcnn_rest_base_FPN_1x.yaml
    SOLVER.IMS_PER_BATCH 2 SOLVER.BASE_LR 0.0025。 然后报错：KeyError: 'Non-existent config key: MODEL.REST'。 请问可以麻烦教一下，怎么运行成功在的detectron2中运行ResT么

    opened by ywx-hub 1
  • 关于类PA的作用

    关于类PA的作用

    你好 很感谢你的工作。有一个问题想请教下：代码中class PA ()的作用是什么？感觉与论文2.4 Position Encoding不对应。因为代码中PA调用是在class PatchEmbed （认为与论文2.3 Patch Embedding相关）class BasicStem （the first patch embedding module）。综上 class PA ()与论文2.3 Patch Embedding相关，不与2.4 Position Encoding相关，但是论文2.4 Position Encoding中公式（8）描述了PA的Conv2d（）和Sigmoid() 。 再次谢谢

    opened by ChenCherry 1
Owner
zhql
Machine Learning Bricklayer
zhql
GitHub
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

null 101 Nov 25, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

Angtian Wang 76 Nov 23, 2022
StyleGAN2-ADA - Official PyTorch implementation

Abstract: Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

NVIDIA Research Projects 3.2k Dec 30, 2022
Official implementation of the ICLR 2021 paper

You Only Need Adversarial Supervision for Semantic Image Synthesis Official PyTorch implementation of the ICLR 2021 paper "You Only Need Adversarial S

Bosch Research 272 Dec 28, 2022
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

Richard Wang 443 Dec 6, 2022
Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

CIPS -- Official Pytorch Implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis Requirements pip install -r requi

Multimodal Lab @ Samsung AI Center Moscow 201 Dec 21, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official PyTorch Implementation of Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity

UnRigidFlow This is the official PyTorch implementation of UnRigidFlow (IJCAI2019). Here are two sample results (~10MB gif for each) of our unsupervis

Liang Liu 28 Nov 16, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Official implementation of Self-supervised Graph Attention Networks (SuperGAT), ICLR 2021.

SuperGAT Official implementation of Self-supervised Graph Attention Networks (SuperGAT). This model is presented at How to Find Your Friendly Neighbor

Dongkwan Kim 127 Dec 28, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.

BiPointNet: Binary Neural Network for Point Clouds Created by Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Li

Haotong Qin 59 Dec 17, 2022
Official code implementation for "Personalized Federated Learning using Hypernetworks"

Personalized Federated Learning using Hypernetworks This is an official implementation of Personalized Federated Learning using Hypernetworks paper. [

Aviv Shamsian 121 Dec 25, 2022
StyleGAN2 - Official TensorFlow Implementation

StyleGAN2 - Official TensorFlow Implementation

NVIDIA Research Projects 10.1k Dec 28, 2022
Old Photo Restoration (Official PyTorch Implementation)

Bringing Old Photo Back to Life (CVPR 2020 oral)

Microsoft 11.3k Dec 30, 2022
Official implementation of "GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators" (NeurIPS 2020)

GS-WGAN This repository contains the implementation for GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators (NeurIPS

null 46 Nov 9, 2022
Official PyTorch implementation of Spatial Dependency Networks.

Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling Đorđe Miladinović   Aleksandar Stanić   Stefan Bauer   Jürgen Schmid

Djordje Miladinovic 34 Jan 19, 2022