I tried to run the command for a test with the 2 pictures provided (I installed pytorch on Anaconda) and I got the following error:

Traceback (most recent call last):
  File "run.py", line 166, in <module>
    moduleNetwork = Network().cuda()
  File "run.py", line 133, in __init__
    self.moduleBasic = torch.nn.ModuleList([ Basic(intLevel) for intLevel in range(6) ])
  File "run.py", line 133, in <listcomp>
    self.moduleBasic = torch.nn.ModuleList([ Basic(intLevel) for intLevel in range(6) ])
  File "run.py", line 100, in __init__
    self.moduleBasic[intConv * 2].weight.data.copy_(torch.utils.serialization.load_lua('./models/modelL' + str(intLevel + 1) + '_' + arguments_strModel  + '-' + str(intConv + 1) + '-weight.t7'))
RuntimeError: The expanded size of the tensor (7) must match the existing size (0) at non-singleton dimension 3

Do you know what could be the reason for this ? thank you !!

It was triggered when running to

tensorFlow = torch.nn.functional.interpolate(input=moduleNetwork(tensorPreprocessedFirst, tensorPreprocessedSecond), size=(intHeight, intWidth), mode='bilinear', align_corners=False)

How can l solve this problem? Is that a cause of low pytorch version? Looking forward to your reply.

Hi, this is an amazing work and saved me from alot of work. I just want to ask that why you have writtenbackwarping outside the Network class while you are also defining preprocess and basic inside the class. is there any theoretical background which I am missing right now?

Note. Actually I am trying to use your code to run on multiple GPUS and this function is creating a problem.

thanks.

return torch.nn.functional.grid_sample(input=tensorInput, grid=(Backward_tensorGrid[str(tensorFlow.size())] + tensorFlow).permute(0, 2, 3, 1),\
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!

Hi! I meet this problem when run run.py. my environment: CUDA 8 + cudnn 7.1+python 3.7. Thank you~ Details as following: Traceback (most recent call last): File "/media/data2/xly/net_code/pytorch-liteflownet-master/run.py", line 377, in tensorOutput = estimate(tensorFirst, tensorSecond) File "/media/data2/xly/net_code/pytorch-liteflownet-master/run.py", line 363, in estimate tensorFlow = torch.nn.functional.interpolate(input=moduleNetwork(tensorPreprocessedFirst, tensorPreprocessedSecond), size=(intHeight, intWidth), mode='bilinear', align_corners=False) File "/home/xly/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, **kwargs) File "/media/data2/xly/net_code/pytorch-liteflownet-master/run.py", line 332, in forward tensorFlow = self.moduleSubpixel[intLevel](tensorFirst[intLevel], tensorSecond[intLevel], tensorFeaturesFirst[intLevel], tensorFeaturesSecond[intLevel], tensorFlow) File "/home/xly/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, **kwargs) File "/media/data2/xly/net_code/pytorch-liteflownet-master/run.py", line 226, in forward tensorFeaturesSecond = Backward(tensorInput=tensorFeaturesSecond, tensorFlow=tensorFlow * self.dblBackward) File "/media/data2/xly/net_code/pytorch-liteflownet-master/run.py", line 61, in Backward return torch.nn.functional.grid_sample(input=tensorInput, grid=(Backward_tensorGrid[str(tensorFlow.size())] + tensorFlow).permute(0, 2, 3, 1), mode='bilinear', padding_mode='zeros') File "/home/xly/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2597, in grid_sample return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum) RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

wget --timestamping http://content.sniklaus.com/github/pytorch-spynet/network-chairs-clean.pytorch wget --timestamping http://content.sniklaus.com/github/pytorch-spynet/network-chairs-final.pytorch

What is the difference between the pre_train models ?

I'm getting the following error when I run: python run.py --model sintel-final --first ./images/first.png --second ./images/second.png --out ./out.flo

Traceback (most recent call last): File "run.py", line 210, in tensorOutput = estimate(tensorInputFirst, tensorInputSecond) File "run.py", line 192, in estimate tensorOutput.resize_(2, intHeight, intWidth).copy_(moduleNetwork(variableInputFirst, variableInputSecond).data[0]) File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in call result = self.forward(*input, **kwargs) File "run.py", line 151, in forward variableFlow = torch.autograd.Variable(data=torch.zeros(variableFirst[0].size(0), 2, math.floor(variableFirst[0].size(2) / 2.0), math.floor(variableFirst[0].size(3) / 2.0)).cuda(), volatile=not self.training) TypeError: torch.zeros received an invalid combination of arguments - got (int, int, float, float), but expected one of:

• (int ... size) didn't match because some of the arguments have invalid types: (int, int, float, float)
• (torch.Size size)

I was able to fix it by casting both math.floor calls to int.

Hello! I tried your demo but got this error. line 151, in forward

variableFlow=torch.autograd.Variable(data=torch.zeros(variableFirst[0].size(0),2,math.floor(variableFirst[0].size(2) / 2.0), math.floor(variableFirst[0].size(3) / 2.0)).cuda(), volatile=not self.training)

TypeError: torch.zeros received an invalid combination of arguments - got (int, int, float, float), but expected one of:

• (int ... size)

I changed to "data=torch.zeros(variableFirst[0].size(0)/2,math.floor....)" but still not work. It seems there's something wrong with the ''size''. I'm new to deep learning, can you give some suggestion?

Thanks!

Thank you for sharing your code. I have a question with this.

What the list ([ 80, 73, 69, 72 ]) mean? https://github.com/sniklaus/pytorch-liteflownet/blob/dae3b132452b05d4258d1da75eaa1021c5b2d720/run.py#L381

Thanks in advance

I was comparing the backwarp function between pytorch-spynet and pytorch-pwc and they are different. Does it matter which one is used? Furthermore, in the model I am developing, I am going to need to warp the image according to the optical flow, do we just use either one of the 2 backwarp functions used in the 2 models. So for example if I used pytorch-spynet then I would use the backwarp of pytorch-spynet, and vice versa for pytorch-pwc?

Thank you for the wonderful work. Relying on this, our team has also pulled out similar work in tensorflow. The architecture is available at: https://github.com/tukilabs/Video-Compression-Net/blob/master/utils/network.py (class SpyNetwork) and the pre-trained model is available at: https://github.com/tukilabs/Video-Compression-Net/blob/master/checkpoints/spymodel.pkl .

Thanks!

Hi, this code is really helpful. I'm wondering how can I predict the second image from the first one + flow.

I tried

second_image_pred = backwarp(first_image, flow). But the result has strange color shift and the motion is not modeled well.

Looking forward to your reply. Thanks!