It's obvious that the code is wrong. You should check the reshape and transpose part in your code. And what is the coef? It has not been mentioned in the paper. And also you didn't import the torch. And the qkv_bias, qk_scale is not used here. Did you really run the multi-head version in your paper?

先reshape（view）再Conv1d和直接11 Conv2d有什么区别，为什么不直接用11 Conv2d呢？

而且这么看的话，EA和ResNet的bottleneck非常相似，都是沙漏状11 Conv2d中间夹一个模块，只不过ResNet夹的是BN-ReLU-33 Conv2d-BN-ReLU，而EA夹的是double normalization

另外，我把normalization从先在N=H*W维度上softmax再在k=S维度上L1 norm改成只在k=S维度上softmax，在简单的任务上性能没有显著差异，不知道作者有没有做相关的消融实验？（论文中没有提及）

Hi, Thanks for releasing the code! Do you think the representation capability of MEA is lower than EA, since the external memories are shared across different heads?

Hello Menhao, I have some question about the difference between code and paper. In the paper, 'Equation (5) is the similarity between the i-th pixel and the j-th rows of M', external attention do the attention between pixels, however in the code I think Conv1d only can do the attention amount only one pixels’ channels, and pixels in F not do attention with pixels in M. And in the paper 'In fact, we find that a small _S_, e.g. 64, works well in experiments.' but in the code d is setted to 64 instead of S.

Hi，Menghao. I run the command: CUDA_VISIBLE_DEVICES=1,3 python transfer_learning.py and want to verify the performance of EANet on Cifar10/100. But after two rounds of iteration, the loss function remains unchanged. And the value of ACC is always 10.000%. The result is shown in the figure below: Could you please help me to solve this problem?

Not able to reproduce semantic segmentation result on PASCAL VOC val with MMSegmentation. I used the code from https://github.com/MenghaoGuo/-EANet/blob/main/model_torch.py and modified nothing despite replacing the backbone with the one in MMSegmentation. After training EANet and PSPNet with several sets of configs, the result is that the mIoU of EANet is always a little bit below PSPNet, e.g. 73 vs 75. Any suggestions?

Thanks for sharing this great work. I was reading the paper and trying to understand the mechanism. In the paper, Fig 4 outcomes caught my attention, and wanted to see how such amazing attention has been generated. Is the generated code already released or will be?

作者您好，关于multi head attention代码中，self.coef=4，这里的coef=4的作用是什么呢？self.trans_dims = nn.Linear(dim, dim * self.coef)的输入输出是不同维度，但原始self attention中的Q经过现性变换前后维度相同，这是为什么呢。

作者您好，最近阅读了你的论文，产生了浓厚的兴趣，仔细阅读了论文和代码，这里有一个疑问。在multi_head_attention_torch.py文件中，外部注意力机制的实现经过了4个线性层，第一个线性层将Q的维度由dim变成dim×4，中间两层是记忆单元维度64，最后输出将dim×4变回dim，但这样做会增加计算量，比自注意力的计算量大，虽然EA的复杂度是线性的。 上面是我的一点疑问，期待你的答复，谢谢。

Intuitively, the memory units serve as prototypes for different patterns, almost play the same role as a convolution kernel (especially 1*1 conv kernel). From the perspective of mathematical operation, In both cases, the dot product between the feature vector and the memory unit/convolution kernel will be the output.

Hence the question comes that, what are the differences between a memory unit and a convolution kernel?

Hi,

I see that the new multi-head version doesn 't use resnet connection, which is different from former version. Nor did you use normalization. How do you think about this ?

Thank you.