Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

Related tags

Deep Learning SA-Net
Overview

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper)

By Qing-Long Zhang and Yu-Bin Yang

[State Key Laboratory for Novel Software Technology at Nanjing University]

Approach

Figure 1: The Diagram of a shuffle attention module.

Comments
  • SA模块添加位置的疑问

    SA模块添加位置的疑问

    您好，读了您发表的SA注意力机制文章，想尝试一下。但是总得不到较好的结果，甚至比不添加注意力精度要低一点点，我想可能是我添加的位置有问题。 我在尝试往YOLOv3的Darknet-53中添加SA注意力，对于位置的选取有些疑问，比如是将SA添加到Residual模块里面或外面，是否对所有Residual都添加SA，以及Groups分组数是否可以修改呢？ 期望您的回复。

    opened by Suncheng2019 3
  • spatial attention的疑问

    spatial attention的疑问

    作者你好，我看了文章以及代码后对提到的spatial attention有两个疑问： 1.得到的attention map似乎不仅仅是1HW维的，而是仍然有通道维，即CHW?似乎不是纯粹的spatial attention。 2. 文章提到在得到spatial map的第一步是用的GN，但事实上，如果设置nn.GroupNorm(channel // (2 * groups), channel // (2 * groups)),是否就等价于InstanceNorm了，这一步和Group没什么关系。

    谢谢！

    opened by XinYu-Andy 2
  • 关于代码的疑惑

    关于代码的疑惑

    @wofmanaf 作者您好 感谢你们的工作，收益很大。 在看代码的过程中存在疑惑，论文中说对特征层进行分组G=64 而代码中SABottleneck部分中 def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1, base_width=64, dilation=1, norm_layer=None) group=1， 而width = int(planes * (base_width / 64.)) * groups 导致后续的width仍然是plane，这么一看应该没有分组啊？ 不知道我的理解是否正确，恳请作者指点！谢谢

    opened by xyl-507 2
  • ValueError: not enough values to unpack (expected 2, got 1)

    ValueError: not enough values to unpack (expected 2, got 1)

    我在yolov5中插入sanet,出现错误 x_0, x_1 = x.chunk(2, dim=1) ValueError: not enough values to unpack (expected 2, got 1)

    因此我打印出他们的形状 def forward(self, x): b, c, h, w = x.shape print(x.shape) x = x.reshape(b // 2, 2, h, w) print(x.shape) x_0, x_1 = x.chunk(2, dim=1) 输出为为（1,256,32,32） （256,1,32,32）

    请问如何解决问题呢？

    opened by fanghua2021 3
  • Getting error

    Getting error

    when using sa_layer in deeplearning model getting following error, kindly help to me, solve this x = x.reshape(b * self.groups, -1, h, w) RuntimeError: shape '[64, -1, 64, 64]' is invalid for input of size 327680 kindly solve this

    opened by devibala242 2
Owner
Qing-Long Zhang
Machine Learning Bricklayer
Qing-Long Zhang
GitHub
Shuffle Attention for MobileNetV3

SA-MobileNetV3 Shuffle Attention for MobileNetV3 Train Run the following command for train model on your own dataset: python train.py --dataset mnist

Sajjad Aemmi 36 Dec 28, 2022
Pytorch implementation of ICASSP 2022 paper Attention Probe: Vision Transformer Distillation in the Wild

Attention Probe: Vision Transformer Distillation in the Wild Jiahao Wang, Mingdeng Cao, Shuwei Shi, Baoyuan Wu, Yujiu Yang In ICASSP 2022 This code is

IIGROUP 6 Sep 21, 2022
Code for the paper "Unsupervised Contrastive Learning of Sound Event Representations", ICASSP 2021.

Unsupervised Contrastive Learning of Sound Event Representations This repository contains the code for the following paper. If you use this code or pa

Eduardo Fonseca 81 Dec 22, 2022
Code for the ICASSP-2021 paper: Continuous Speech Separation with Conformer.

Continuous Speech Separation with Conformer Introduction We examine the use of the Conformer architecture for continuous speech separation. Conformer

Sanyuan Chen (陈三元) 81 Nov 28, 2022
Neural networks applied in recognizing guitar chords using python, AutoML.NET with C# and .NET Core

Chord Recognition Demo application The demo application is written in C# with .NETCore. As of July 9, 2020, the only version available is for windows

Andres Mauricio Rondon Patiño 24 Oct 22, 2022
An implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks in PyTorch.

Neural Attention Distillation This is an implementation demo of the ICLR 2021 paper Neural Attention Distillation: Erasing Backdoor Triggers from Deep

Yige-Li 84 Jan 4, 2023
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

Matthew Macy 606 Dec 21, 2022
Code to reproduce the experiments from our NeurIPS 2021 paper " The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective"

Code To run: python runner.py new --save <SAVE_NAME> --data <PATH_TO_DATA_DIR> --dataset <DATASET> --model <model_name> [options] --n 1000 - train - t

Geoff Pleiss 5 Dec 12, 2022
RGBD-Net - This repository contains a pytorch lightning implementation for the 3DV 2021 RGBD-Net paper.

[3DV 2021] We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network.

Phong Nguyen Ha 4 May 26, 2022
This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

CPC_DeepCluster This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEEC

LEAP Lab 2 Sep 15, 2022
Official implementation of FCL-taco2: Fast, Controllable and Lightweight version of Tacotron2 @ ICASSP 2021

FCL-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech synthesis (ICASSP 2021) Paper | Demo Block diagram of FCL-taco2, where the decode

Disong Wang 39 Sep 28, 2022
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

Haoyu Chen 71 Dec 30, 2022
The implementation of ICASSP 2020 paper "Pixel-level self-paced learning for super-resolution"

Pixel-level Self-Paced Learning for Super-Resolution This is an official implementaion of the paper Pixel-level Self-Paced Learning for Super-Resoluti

Elon Lin 41 Dec 15, 2022
Implementation of Uniformer, a simple attention and 3d convolutional net that achieved SOTA in a number of video classification tasks

Uniformer - Pytorch Implementation of Uniformer, a simple attention and 3d convolutional net that achieved SOTA in a number of video classification ta

Phil Wang 90 Nov 24, 2022
This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

Maurice Weiler 59 Dec 9, 2022
An implementation of the research paper "Retina Blood Vessel Segmentation Using A U-Net Based Convolutional Neural Network"

Retina Blood Vessels Segmentation This is an implementation of the research paper "Retina Blood Vessel Segmentation Using A U-Net Based Convolutional

Srijarko Roy 23 Aug 20, 2022
Code for our paper at ECCV 2020: Post-Training Piecewise Linear Quantization for Deep Neural Networks

PWLQ Updates 2020/07/16 - We are working on getting permission from our institution to release our source code. We will release it once we are granted

null 54 Dec 15, 2022