First off, thank you for your amazing port, @1zb!

How does num_deformable_groups work? I was wondering if it is splitting the offsets, thus each group of input channels will have a given set of offsets (which may be different from other groups). For instance, if we have an input of 6 channels (1, 6, H, W), and num_deformable_groups=2, then we would have our input split between two tensors of (1, 3, H, W) and (1, 3, H, W), each with its respective instance of offset mask (1, 3, kH, kW). Is that correct?

Hi: When I ran CC=g++ python build.py, I got errors of cffi.error.VerificationError: CompileError: command 'g++' failed with exit status 1. Could you give me some suggestions? Thank you. @1zb The error details are as follows:

Traceback (most recent call last):
  File "/home/sean/anaconda3/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/home/sean/anaconda3/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/home/sean/anaconda3/lib/python3.6/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/home/sean/anaconda3/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'g++' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 51, in _build
    dist.run_command('build_ext')
  File "/home/sean/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/sean/anaconda3/lib/python3.6/distutils/command/build_ext.py", line 533, in build_extension
    depends=ext.depends)
  File "/home/sean/anaconda3/lib/python3.6/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/sean/anaconda3/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'g++' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "build.py", line 36, in <module>
    ffi.build()
  File "/home/sean/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 189, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/__init__.py", line 111, in _build_extension
    outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/home/sean/anaconda3/lib/python3.6/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'g++' failed with exit status 1

First, thanks for the great work. I found there's possible memory leak in the code. To reproduce the problem, change the test.py as below:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from torch.autograd import gradcheck
from torch.nn.modules.utils import _single, _pair

from modules import ConvOffset2d

num_deformable_group = 1

N, inC, inH, inW = 1, 3, 512, 512
outC, outH, outW = 4, 512, 512
kH, kW = 3, 3

conv = nn.Conv2d(inC, num_deformable_group * 2 * kH * kW, kernel_size=(kH, kW), stride=(1,1), padding=(1,1), bias=False).cuda()
conv_offset2d = ConvOffset2d(inC, outC, (kH, kW), stride=1, padding=1).cuda()

for epoch in range(500):
    inputs = Variable(torch.randn(N, inC, inH, inW).cuda())
    offset = conv(inputs)
    output = conv_offset2d(inputs, offset)
    output.backward(output.data)
    print(output.size())

Code will crash for OOM while doing the backward()

Thanks for your code firstly. In this implementation, the spatial resolution of offset should be the same with output feature map. But in the original paper, the spatial resolution of offset is the same with input feature map. Does this have an effect on the performance?

run nvcc -c -o deform_conv_cuda_kernel.cu.o deform_conv_cuda_kernel.cu -x cu -Xcompiler -fPIC -std=c++11 happens error，nvcc fatal : Cannot find compiler 'cl.exe' in PATH

I ran the sh make.sh and then CC=g++ python build.py. It seems that It is building fine, however after running the test.py I have got error. Please note that in the _ext/deform_conv I have only an init.py and _deform_conv. I really appreciate your help.

sh make.sh CC=g++ python build.py Including CUDA code. /home/mika/wd/deformable-convolution-pytorch-master generating /tmp/tmp_sneiacp/_deform_conv.c setting the current directory to '/tmp/tmp_sneiacp' running build_ext building '_deform_conv' extension creating home creating home/mika creating home/mika/wd creating home/mika/wd/deformable-convolution-pytorch-master creating home/mika/wd/deformable-convolution-pytorch-master/src g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/v/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c _deform_conv.c -o ./_deform_conv.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.c -o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ g++ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/mika/.conda/envs/py36/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/mika/.conda/envs/py36/include/python3.6m -c /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.c -o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.o cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ gcc -pthread -shared -B /home/mika/.conda/envs/py36/compiler_compat -L/home/mika/.conda/envs/py36/lib -Wl,-rpath=/home/mika/.conda/envs/py36/lib -Wl,--no-as-needed -Wl,--sysroot=/ ./_deform_conv.o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv.o ./home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda.o /home/mika/wd/deformable-convolution-pytorch-master/src/deform_conv_cuda_kernel.cu.o -o ./_deform_conv.so

python test.py Traceback (most recent call last): File "test.py", line 6, in from modules import ConvOffset2d File "/home/v/wd/deformable-convolution-pytorch-master/modules/init.py", line 1, in from .deform_conv import ConvOffset2d File "/home/mika/wd/deformable-convolution-pytorch-master/modules/deform_conv.py", line 7, in from functions import conv_offset2d File "/home/mika/wd/deformable-convolution-pytorch-master/functions/init.py", line 1, in from .deform_conv import conv_offset2d File "/home/mika/wd/deformable-convolution-pytorch-master/functions/deform_conv.py", line 5, in from _ext import deform_conv File "/home/mika/wd/deformable-convolution-pytorch-master/_ext/deform_conv/init.py", line 3, in from ._deform_conv import lib as _lib, ffi as _ffi ImportError: /home/mika/wd/deformable-convolution-pytorch-master/_ext/deform_conv/_deform_conv.so: undefined symbol: __cudaPopCallConfiguration

Thinking about deformable convolutions, some things I found different interpretations of: -) Does the offset change for each individual k x k kernel or is it fixed for the whole image? Would this mean that pixels could potentially overlap? -) Is the same offset then applied for each input layer, ie. AxBxC where C might be any number of filters. -) During inference, keeping the offset generating layers in the network, each k x k kernel would experience an individual offset, or would the offset be the same for the whole image?