I was investigating differences in results of this package and of the original Caffe version. Things found so far:

1. Rfft can be used directly, it is provided by pytorch_fft
2. Complex product should be used here: https://github.com/DeepInsight-PCALab/CompactBilinearPooling-Pytorch/blob/master/CompactBilinearPooling.py#L91, the tf.multiply does complex product, original caffe version does complex product
3. Output should be mutliplied by output_dim in order to achieve full equivalence with original caffe version

Hi, thanks for your job! lt looks like there is no module named _th_fft when follow the command as you said. Do you have any idea about it? Thanks a lot~ ` dl@dl:~/wxptest$ sudo python wxp.py

Traceback (most recent call last): File "wxp.py", line 3, in from CompactBilinearPooling import CompactBilinearPooling File "/home/dl/wxptest/CompactBilinearPooling.py", line 6, in import pytorch_fft.fft.autograd as afft File "/home/dl/wxptest/pytorch_fft/init.py", line 1, in from . import fft File "/home/dl/wxptest/pytorch_fft/fft/init.py", line 1, in from .fft import * File "/home/dl/wxptest/pytorch_fft/fft/fft.py", line 3, in from .._ext import th_fft File "/home/dl/wxptest/pytorch_fft/_ext/th_fft/init.py", line 3, in from ._th_fft import lib as _lib, ffi as _ffi ImportError: No module named _th_fft `

when I install pytorch_fft,the problem is： error in pytorch_fft setup command: 'C:\Users\14375\AppData\Local\Temp\pip-install-6vh12s8w\pytorch-fft\build.py:ffi' must be of the form 'path/build.py:ffi_va riable' Including CUDA code. ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

According to the existing code beginning here https://github.com/DeepInsight-PCALab/CompactBilinearPooling-Pytorch/blob/56e23d01e687dad067dfeb9a07bb4d012430e1df/CompactBilinearPooling.py#L92 the product of two complex numbers(a+ib, x+iy) is {(ax-by) + i(ax+by)}.

   `temp_rr, temp_ii = fft1_real.mul(fft2_real), fft1_imag.mul(fft2_imag)
    fft_product_real = temp_rr - temp_ii
    fft_product_imag = temp_rr + temp_ii

    cbp_flat = afft.Ifft()(fft_product_real, fft_product_imag)[0]`

However the correct product is {(ax-by) + i(ay+bx)} in which case the following product is wrong.

Correct me if I am wrong.

I am getting this particular error while running sample test given in readme.

  File "CompactBilinearPooling/test.py", line 14, in <module>
    out = layer(bottom1, bottom2) 
  File "anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "CompactBilinearPooling/CompactBilinearPooling.py", line 91, in forward
    fft1_real, fft1_imag = afft.Fft()(sketch_1, Variable(torch.zeros(sketch_1.size())).cuda())
  File "anaconda3/lib/python3.6/site-packages/pytorch_fft/fft/autograd.py", line 16, in forward
    return fft(X_re, X_im)
  File "anaconda3/lib/python3.6/site-packages/pytorch_fft/fft/fft.py", line 25, in fft
    raise NotImplementedError
NotImplementedError

Now this is due to this function fft.py:

    if 'Float' in type(X_re).__name__ :
        f = th_fft.th_Float_fft1
    elif 'Double' in type(X_re).__name__: 
        f = th_fft.th_Double_fft1
    else: 
        raise NotImplementedError
    return _fft(X_re, X_im, f, 1)

Because inputs to fft() are tensors, type(sketch_1) does not include 'Float' or 'Double'. Any help is appreciated.

Thanks for your working, when I apply your code, I meet the situation that model.parameters() is empty and fail to creat optimizer. Model is compactbilinearpooling(1536,1536,400). Am I wrong? Thanks for answering

Thank you for your job! I have some question, if x = (4,512,64,64), y = (4,512,64,64), a=CompactBilinearPooling(x,y) ， what the shape of a ? And How I can make the a‘s shape is (batch_size,channels,height,width) which the same as x and y？

rand_h_1 = np.random.randint(output_dim, size=self.input_dim1) and rand_s_1 = 2 * np.random.randint(2, size=self.input_dim1) - 1 in the code seem to be different from the setting of the paper. In the paper, hk is {1,2,...,k}, sk is {+1 ,-1}.

For multi GPU, it outputs:

RuntimeError: arguments are located on different GPUs at /pytorch/torch/lib/THC/generic/THCTensorMathBlas.cu:236

How to fix it?