Hi, I have read the paper“Deformable Convolutional Networks”and your pytoch-deform-conv code. Then I think that the size of offset maybe bckenel[0]*kenel[1]hw. Maybe I understand it wrong. Can you explain it?

Hello oeway

Firstly, thanks for sharing your pytorch version code of deformable convolution. When I tried to port your code to another code which uses CUDA mode, it gives me an error saying like below.

'Type torch.cuda.FloatTensor doesn't implement stateless method range'

However, when I change it to CPU mode, there is no error.

It seems like 'torch.range' does not fit on GPU mode in torch. Then what should be the solution?

Thanks in advance

Hi, why train first on the convnet and only after do finetuning on the deformable convnet? why not straight traing on deformable (with freezing the ofsets as written in the paper)

https://github.com/oeway/pytorch-deform-conv/blob/e7a664f1e868f254d758e1cde11d82ea66972aa4/torch_deform_conv/layers.py#L33 Your implementation is not the same as the original paper described in section 2.1. Offset output channel dimension should be 2 x N, where N is the conv kernel size k*k in 2D case. That is, the offset is shared across the channel dimension. You could refer to the implementation of https://github.com/ChunhuanLin/deform_conv_pytorch

Hi, when I run scale_mnist.py, I get into this trouble. I find that process exits when it imports Keras. If someone knows how to solve this problem, please tell me. Thanks! ^.^

Hi oeway,

Any chance you can help me understand your code? On this line, you index the input with a detached variable, so I'm wondering how you propagate the gradient backward through the vals_lt, etc.. It seems like mapped_vals would not have any parent nodes with gradients? Does that make sense? When I try to do a similar thing here for a spatial transformer network, it gives me a no nodes require gradients error.

Do you get around this by freezing the entire network? I feel like you would get the same error if the network wasn't frozen. Any insight you can provide into this would be appreciated.

EDIT: Ok, I get that the gradient propagates through the coords_offset_lt value... can you describe where you got this interpolation algorithm from? Thanks :)

The issue proposed in issue 4. I test the situation of : both network is training on the dataset (not deform mnist), the test accuracy

---original net---
test_data:             99.03%
test_scaled_data: 62.22%
---deformable net---
test_data:              98.67%
test_scaled_data: 54.43%

I think the train phase should use undeform data, this can show the advantage of the deformable CNN's advantage.

The issue proposed in issue 4. I test the situation of : both network is training on the dataset (not deform mnist), the test accuracy

---original net---
test_data:             99.03%
test_scaled_data: 62.22%
---deformable net---
test_data:              98.67%
test_scaled_data: 54.43%

I think the train phase should use undeform data, this can show the advantage of the deformable CNN's advantage.

I trained your deform-conv on the origin data. It gets 70% on origin test and 89% on scaled data, while the the normal CNN model trained on origin data gets 99% on origin test and 64% on scaled data. The deform-conv may not work. So I try to delete two of the deform-conv, keep one left. It gets 93% on origin test and 97% on scaled data. Your deform-conv may not work. I think you should check it.

I am a little bit confused how the weighting is done of each input element of the deformed kernel in your implementation.

In your implementation it looks like you are first calculating the offsets and with these rearrange the input in the ConvOffset2D module (so ConvOffset2D just outputs the rearranged features according to the learned offsets). After that, on the rearranged feature maps you apply a normal Conv2D, which should then do the actual weighting for the deformed kernel, right?

Is this the same as in the original paper where I understand it that way that the weighting is applied directly to the deformed input elements (kernel elements with offsets) , like

sum_p_n[ w(p_n) * x(p_0 + p_n + p_delta)]

I don't see if this is the same or if this has a different assumption on deformable confolutions than the original paper.

Hello,

I am confused about the shape of offset. The paper mentions: "The grid R defines the receptive field size and dilation. For example,R = {(−1,−1),(−1,0),...,(0,1),(1,1)}. In deformable convolution, the regular grid R is aug- mented with offsets {∆pn |n = 1, ..., N }, where N = |R|. The output offset fields have the same spatial resolution with the input feature map. The channel dimension 2N corresponds to N 2D offsets." So, I think the shape of offset field would be [2*9, H, W] if 3x3 kernel is used. While in your implementation, the shape of offset seems to be [batch_size, 2*n_channels, H, W]?

Hello. Thanks for sharing the code. I have a question about the implementation of offset, in [https://github.com/oeway/pytorch-deform-conv/blob/master/torch_deform_conv/deform_conv.py#L182] the code :

offsets = offsets.view(batch_size, -1, 2)

the input tensor offsets in b * (2c) * h * w after first normal conv, I think the offset of defom-conv is the output channels, therefore is the code should be ? :

offsets = offsets.view(b, 2*c, h, w)
offsets = offsets.permute(0, 2, 3, 1)