Code for our CVPR2021 paper coordinate attention

Qibin (Andrew) Hou

Last update: Jan 5, 2023

Related tags

Deep Learning CoordAttention

Overview

Coordinate Attention for Efficient Mobile Network Design (preprint)

This repository is a PyTorch implementation of our coordinate attention (will appear in CVPR2021).

Our coordinate attention can be easily plugged into any classic building blocks as a feature representation augmentation tool. Here (pytorch-image-models) is a code base that you might want to train a classification model on ImageNet.

Note that the results reported in the paper are based on regular training setting (200 training epochs, random crop, and cosine learning schedule) without using extra label smoothing, random augmentation, random erasing, mixup. For specific numbers in ImageNet classification, COCO object detection, and semantic segmentation, please refer to our paper.

Comparison to Squeeze-and-Excitation block and CBAM

(a) Squeeze-and-Excitation block (b) CBAM (C) Coordinate attention block

How to plug the proposed CA block in the inverted residual block and the sandglass block

(a) MobileNetV2 (b) MobileNeXt

Some tips for designing lightweight attention blocks

SiLU activation (h_swish in the code) works better than ReLU6
Either horizontal or vertical direction attention performs the same to the SE attention
When applied to MobileNeXt, adding the attention block after the first depthwise 3x3 convolution works better
Note sure whether the results would be better if a softmax is applied between the horizontal and vertical features

Object detection

We use this repo (ssdlite-pytorch-mobilenext).

Semantic segmentation

We use this repo. You can also refer to mmsegmentation alternatively.

Citation

You may want to cite:

@inproceedings{hou2021coordinate,
  title={Coordinate Attention for Efficient Mobile Network Design},
  author={Hou, Qibin and Zhou, Daquan and Feng, Jiashi},
  booktitle={CVPR},
  year={2021}
}

@inproceedings{sandler2018mobilenetv2,
  title={Mobilenetv2: Inverted residuals and linear bottlenecks},
  author={Sandler, Mark and Howard, Andrew and Zhu, Menglong and Zhmoginov, Andrey and Chen, Liang-Chieh},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={4510--4520},
  year={2018}
}

@inproceedings{zhou2020rethinking,
  title={Rethinking bottleneck structure for efficient mobile network design},
  author={Zhou, Daquan and Hou, Qibin and Chen, Yunpeng and Feng, Jiashi and Yan, Shuicheng}
  booktitle={ECCV},
  year={2020}
}

@inproceedings{hu2018squeeze,
  title={Squeeze-and-excitation networks},
  author={Hu, Jie and Shen, Li and Sun, Gang},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={7132--7141},
  year={2018}
}

@inproceedings{woo2018cbam,
  title={Cbam: Convolutional block attention module},
  author={Woo, Sanghyun and Park, Jongchan and Lee, Joon-Young and Kweon, In So},
  booktitle={Proceedings of the European conference on computer vision (ECCV)},
  pages={3--19},
  year={2018}
}

Comments

A question about the reduction's best choice

Hello, I recently added your CA in target detection and experimented with the reduction to 32,16,8.The model was evaluated by MAP_0.5, Precison, and Recall related indicators.It is found that the situation where r=16 is the optimal one, 8 is better than 32 but weaker than 16. However, your paper said ''This demonstrates that adding more parameters by reducing the reduction ratio matters for improving the model performance''.However, the map, precison, and recall values of r=8 are weaker than those of r=16. Hope to get your reply as soon as possible,Thank you！

opened by woson-L 4
TypeError: must be real number, not NoneType

Hi! When I add CA module to the backbone, I got TypeError: "must be real number, not NoneType". The error occurs in nn.AdaptiveAvgPool2d((None, 1)). How to solve the problem?

opened by huanmx 3
Is there a performance improvement for CA

I added the CA module to GhostNet, and the Map dropped. At the same time, adding the CA module to the Neck's Concat function also decreased the accuracy. Why? But, adding SEnet can increase the accuracy.

opened by fanghua2021 3
Where is the best position to insert the CoordAttention？

Could you give me some advice about where to insert the CoordAttention?Did it will perform best on the begin of the net ,on the end of the net or before the Conv?

opened by Fly-Pluche 2
What's the meaning of the Concatenation and Split operation?

Hey, Good Work! However, I still don't understand the meaning of concatenation. After X AVG POOL and Y AVG POOL, we got two feature maps whose shapes like (C, H, 1) and (C, 1, W), and then you choose to concatenate them in the spatial dimension and get feature map shape like (C, H+W, 1), after that you apply a 1x1 convolution filter to it. Since the conv filter is applied to the channel dimension and it has kernel size of 1x1, there is actually no information exchange in the spatial dimension like the Spatial Attention Module in CBAM. The only spatial information exchange happens in the BatchNorm in CoordAttention, which I think is trivial. Actually I think the Concatenation and Split operation is redundant, why you choose such a design?

opened by BluebirdStory 2
about code bug

when i use the code in my networks ,i met an error like this:ModuleNotFoundError: No module named 'model.attention.CoordAttention',what couse this problem?

opened by 123456789live 0
A Need For Mobilenext+CA pretrained model?

Dear author,thanks for watching my issue.Due to the limitation of computer computing resources，I can't pre-train the Mobilenext+CA model.Do you have any pre-trained model for me to keep the reaserch? Appreciate

opened by liukangji 1

Owner

Qibin (Andrew) Hou

Research fellow at NUS.

GitHub

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Adam-NSCL This is a PyTorch implementation of Adam-NSCL algorithm for continual learning from our CVPR2021 (oral) paper: Title: Training Networks in N

34 Dec 21, 2022

Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

Invariant Point Attention - Pytorch Implementation of Invariant Point Attention as a standalone module, which was used in the structure module of Alph

113 Jan 5, 2023

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

71 Dec 30, 2022

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.

TBE The source code for our paper "Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Le

150 Dec 28, 2022

Code for the paper "Graph Attention Tracking". (CVPR2021)

SiamGAT 1. Environment setup This code has been tested on Ubuntu 16.04, Python 3.5, Pytorch 1.2.0, CUDA 9.0. Please install related libraries before r

122 Dec 24, 2022

Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

OTA: Optimal Transport Assignment for Object Detection This project provides an implementation for our CVPR2021 paper "OTA: Optimal Transport Assignme

217 Jan 3, 2023

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

README.md shall be finished soon. WSSGG 0 Overview 1 Installation 1.1 Faster-RCNN 1.2 Language Parser 1.3 GloVe Embeddings 2 Settings 2.1 VG-GT-Graph

35 Nov 20, 2022

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

59 Dec 9, 2022

Code for our CVPR2021 paper coordinate attention

Related tags

Overview

Coordinate Attention for Efficient Mobile Network Design (preprint)

Comparison to Squeeze-and-Excitation block and CBAM

How to plug the proposed CA block in the inverted residual block and the sandglass block

Some tips for designing lightweight attention blocks

Object detection

Semantic segmentation

Citation

Comments

A question about the reduction's best choice

TypeError: must be real number, not NoneType

Is there a performance improvement for CA

Where is the best position to insert the CoordAttention？

What's the meaning of the Concatenation and Split operation?

about code bug

A Need For Mobilenext+CA pretrained model?

Owner

Qibin (Andrew) Hou

PyTorch implementation of our Adam-NSCL algorithm from our CVPR2021 (oral) paper "Training Networks in Null Space for Continual Learning"

Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.

Code for the paper "Graph Attention Tracking". (CVPR2021)

Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Progressive Coordinate Transforms for Monocular 3D Object Detection

offical implement of our Lifelong Person Re-Identification via Adaptive Knowledge Accumulation in CVPR2021

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Code for CVPR2021 paper "Robust Reflection Removal with Reflection-free Flash-only Cues"

PyTorch code for the paper "Curriculum Graph Co-Teaching for Multi-target Domain Adaptation" (CVPR2021)

Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.

Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.