Hi, I came across your work and thought it was a very interesting concept. Currently, the network takes in fixed input sizes. But is there a way for there to be flexible input sizes? I realize the main constraint here is the following line where the complex weight is defined during initialization time: self.complex_weight = nn.Parameter(torch.randn(h, w, dim, 2, dtype=torch.float32) * 0.02) Is there a way to modify this line so that we can have inputs of different sizes? Thanks

Hi, interesting work! I wonder how the Complexity (FLOPs) for global filter in Table.1 is calculated. Since the conjugate symmetric for real signals, we have:

case1: consider the conjugate symmetric. RFFT: HWD/2 * log2(HW) Global Filter: HWD/2 IRFFT: HWD/2 * log2(HW)

Thus, the total Complexity (FLOPs) for global filter is: HWD * log2(HW) + HWD/2. Is it right?

case2: not consider the conjugate symmetric. RFFT: HWD * log2(HW) Global Filter: HWD IRFFT: HWD * log2(HW)

Thus, the total Complexity (FLOPs) for global filter is: 2HWD * log2(HW) + HWD. Is it right?

Which is right?

Hello, thanks for your nice work!

I wonder what does the option fp32fft do. In my experiments the input and output to the fft function are already torch.float32, so I'm not sure why there is an option for converting to fp32. Thanks in advance

Hi, Rao Thank you for your great work! When measuring GFNet's advertising robustness through FGSM and PGD, can I know specific conditions and hyperparameters? It would be even better if I could get the code you used!

Hi! very interesting work!

How is Params calculated? Do you use profile ?

I have noticed that you use a script to calculate memory and flops. Can you share the script? Many thanks goes to the author~

Thank you for your excellent work.

I'm curious why w is set to w = h // 2 + 1, or just the experiment proves that it is better to set w = h // 2 + 1 in this way

Hello, thanks for your great work!

In your figure and code, there is no skip connection after the global filter layer.

This is different from original transformer implementation, which has 2 skip connections in a single block (each for self-attention layer and FFN layer)

For example, original transformer uses blocks like

x = x + SA(x)
x = x + FFN(x)

But, global filter network uses below block

x_ = Global_Filter(x)
x = x + FFN(x_)

Is there any reason for adopting the current block architecture?

Thanks,

Hi, Thank you for your excellent work. I have some questions concerning 3D configuration of GF-Net. I extend your model to a 3D version by introducing 3D FFT and IFFT to conduct global filter learning, and test on the 3D data classification (Point Cloud / Volumetric data). However, the over-fitting problem occurs (Traning dataset 97% acc. Testing dataset 75% acc.). Could you provide some advice on how to train such a model. (I have tried Dropout with different ratios). Thank you~

Hi, Yongming

What is image size did you use for training and validation on ADE20K?

I noticed that PVT used 512x512 for training and a different scale for testing. However, as the parameters of Global Filter are related to the image size, how do you deal with the scale change?

Thanks in advance.

Hello, thanks for your nice work!

I test the FLOPs of GFNet using the fvcore library as following:

    from fvcore.nn import flop_count
    model_mode = model.training
    model.eval()
    fake_input = torch.rand(1, 3, 224, 224)
    flops_dict, *_ = flop_count(model, fake_input)
    count = sum(flops_dict.values())
    model.train(model_mode)
    print("fvcore FLOPs: {:.3f} G".format(count))

For gfnet-h-b model, I got the result: 8.547 G. It is slightly higher than what you mentioned in the paper, i.e., 8.4G.

My concerns are:

1. How do you get the FLOPs value?

2. From, the fvcore log, I noticed that:

Unsupported operator aten::fft_irfft2 encountered 36 time(s)

I.e., the FLOPs of the fft_irfft2 operator are not taken into account.
I wonder if you consider this operator when calculating the FLOPs?

If not, I think it would be better to consider it because this is the core operator that replaces self-attention.

Please let me know if I missed something.

Thanks.

class GlobalFilter(nn.Module): def init(self, dim, h=14, w=8): super().init() self.complex_weight = nn.Parameter(torch.randn(h, w, dim, 2, dtype=torch.float32) * 0.02)

def forward(self, x):
    B, H, W, C = x.shape
    x = torch.fft.rfft2(x, dim=(1, 2), norm='ortho')
    weight = torch.view_as_complex(self.complex_weight)
    x = x * weight
    x = torch.fft.irfft2(x, s=(H, W), dim=(1, 2), norm='ortho')
    return x

Thank you very much for your work. I have some questions. What's meaning of "h=14, w=8", "s=(H, W), dim=(1, 2)".

Thank you very much for your work, but when I realized the visualization of the frequency domain filter, the image is different from the one you provided. And I would like to ask how to realize the visualization of the spatial domain