Hi,

I wonder to know whether GMA could handle the case that subtitle accompany with large motion which is common in the movie? Would the subtitle be kept well in the interpolation result?

Thanks for making your source code public. Could you please share your code for visualizing the attention map in Figure 6 or guide me on how to obtain it?

Hi, I have run the first two stages in your train.sh, which are chairs and things. But I only get 1.35 and 2.83 EPE on Sintel's clean and final pass. I want to know how can I get the same result in your paper, that is 1.30 and 2.74. Is it achieved by training multiple times to get the optimal value?

Hi! great work! When I tested sintel-test-final dataset and created a create_sintel_submission, It happened" IndexError: index 0 is out of bounds for axis 0 with size 0" on a pair of images, why?

When I input two images of size 768 x 1856 to the model, I got the error below: einops.EinopsError: Error while processing rearrange-reduction pattern "(y v) d -> y () v d". Input tensor shape: torch.Size([25600, 128]). Additional info: {'y': 232}. Shape mismatch, can't divide axis of length 25600 in chunks of 232 Any idea why this happens?

Thank you for your concise and efficient work! I would like to ask a question about the impact of the transformer head's number. I found no ablation experiments for this variable in the paper you published, and you set it to 1 in your code. May I ask if you have conducted relevant experiments on this variable, and whether the performance will be improved if the number is increased?

Hey @zacjiang ,

Thank you for sharing your work! @zacjiang, I was looking to evaluate the pre-trained model on the KITTI test set. I have completed the repository set up, and got it running according to instructions mentioned on GitHub. I was able to reproduce the results mentioned in the paper in Table 2 for the KITTI train dataset.

But When I run it for the KITTI test, the execution of evaluation.py fails. This is because in evaluation.py file, It expects 4 outputs from the data loader, https://github.com/zacjiang/GMA/blob/2f1fd29468a86a354d44dd25d107930b3f175043/evaluate.py#L348-L355. Whereas if the split is test then according to the dataset file here: https://github.com/zacjiang/GMA/blob/2f1fd29468a86a354d44dd25d107930b3f175043/core/datasets.py#L38-L46. It returns only three values.

Then in that case how do i get numbers for KITTI Test split.

Regards, Nitin Bansal

Hi, it is a very nice work! I would like to ask questions about kitti submission. When I get 'kitti_submission' folder, how can I create the right ZIP file. The tips on the Kitti website are as follows: For the optical flow task, do we only need to include the 'flow' folder in the submitted folder?So just rename your procedurally generated 'kitti_submission' folder to 'flow' and compress it into a ZIP file?

Hi, file named 'things_val_test_set.txt' cannot be found in in core/datasets.py, which leads to failure when validate flyingthings3d dataset. Please provide this file, and explain the source of that. Thanks.

Hi, thank you for your great work.

I want to test the GMA on real-world images (ie, not synthetic ones). Could you tell me which one of the four checkpoints (chairs, kitti, sintel, things) is expected to generalize well on real-world images?

I wonder how to accurately count floating point operations per second of GMA model. The open interface basically only counts the convolution , for those special operations in the model, are there any good statistical methods? In other words, how do you do that?

Hello, Thank you very much for sharing your precious work with us. I had two questions regarding the code.

1. In gma.py, line 60, the query tensor is scaled by self.scale = dim_head ** -0.5. Why is that necessary? I would also be thankful if you explain why you set the value to dim_head ** -0.5.
2. In the same file, line 113, motion features are added to the attention output tensor. Could you please give some insights on that as well?

Thanks a lot. Azin

It says,"we project the context feature map to a query feature map and a key feature map. We then take the dot product of the two feature maps and a softmax to obtain an attention matrix" but in network.py line 99,I just found "attention = self.att(inp)". this is what puzzled me.

Thanks for the great code ! I try to reproduce the Sintel and KITTI test results reported in the paper. However, I got 1.58 on Sintel clean, 2.64 on Sintel final for GMA(our), and 5.14 on KITTI for GMA(p only). The results seem worse than those reported in the paper (1.39 on Sintel clean, 2.47 on Sintel final for GMA(our), and 4.93 on KITTI for GMA(p only)). Is it because you find the best iteration checkpoint on the validation set, while I use the last iteration checkpoint? If so, may I know the validation set you choose?

I was using HD1k when doing Sintel finetuning, just as GMA does. I'm surprised that it only consists of grayscale images; that means there's a big domain gap between HD1k and other training sets. Wonder if I remove HD1k, would the model be trained better? My training without HD1k is ongoing, and seems the loss on the training data is much smaller, and accuracy is higher. Would update when it finishes.

Hi, I'm trying to reproduce your result on sintel benchmark. I notice that you use 'C + T + S/K (+ H)' in the experiment table of the paper. To my knowledge, referring to the RAFT paper, C+T+S/K means when you train the sintel stage, you only use C+T+S. I have no idea what is the (+H), even with the explain “S/K (+ H)” refers to methods that are fine-tuned on the Sintel and KITTI datasets, with some also fine-tuned on the HD1K dataset. '. What does thewith some' mean? Could you please detail the training schedule you used in the Sintel submission? Is it the C+T+S+H?