Hi.

Thanks for the nice work! I'm trying to reproduce the results for KITTI dataset reported in Table 4. Can you please tell me which pretrained weights and the hyperparameters I should use in order to obtain the best results?

Also, I'm planning to evaluate GMFlow on HPatches and ETH3D as well. For this, can you tell me , in order to obtain the best results, which weights and hyperparameters I should select?

I will be waiting for your reply! Thanks!

I'm curious to use GMFlow for stereo matching task. I noticed the closed issue #13 where you suggested to replace 2D cross-attention in Transformers with 1D cross-attention and 2D global matching with 1D global matching.

Using Stereo-RAFT 1D correlation as a model I did manage to implement somehow 1D global matching but not sure at all if it is right or wrong (didn't manage to include pred_bidir_flow parameter).

def coords_grid(b, h, w):
    y, x = torch.meshgrid(torch.arange(h), torch.arange(w))  # [H, W]
    stacks = [x, y]
    grid = torch.stack(stacks, dim=0).float()  # [2, H, W] or [3, H, W]
    grid = grid[None].repeat(b, 1, 1, 1)  # [B, 2, H, W] or [B, 3, H, W]
    return grid

def global_correlation_softmax_1d(feature0, feature1):
    # global correlation
    b, c, h, w = feature0.shape

    feature0 = feature0.permute(0, 2, 3, 1)
    feature1 = feature1.permute(0, 2, 1, 3)

    corr = torch.matmul(feature0, feature1) / (c ** 0.5) # [B, H, W, W]

    # flow from softmax
    init_grid = coords_grid(b, h, w).to(corr.device)  # [B, 2, H, W]
    grid = init_grid.permute(0, 2, 3, 1)  # [B, H*W, 2]

    prob = F.softmax(corr, dim=-1)  # [B, H, W, W]

    correspondence = torch.matmul(prob, grid).permute(0, 3, 1, 2)  # [B, 2, H, W]

    # when predicting bidirectional flow, flow is the concatenation of forward flow and backward flow
    flow = correspondence - init_grid

    return flow

For the 2D cross-attention replacement with 1D variant I don't know exactly which function I should modify. I assume that modifications should be done in single_head_split_window_attention() function but no idea how.

I also noticed that the results of GMStereo is added to Middlebury stereo evalutation. Are you planning to release codes related to that project (this would solve automatically my issues related to modifications)?

您好，您在计算相似度矩阵时是直接将特征向量进行点积运算得到相似性矩阵。我把特征向量进行了归一化处理（每个像素点特征向量的模归一化为1），然后将归一化后的特征向量进行点积运算得到相似性矩阵，相似性矩阵没有再除以c**0.5。理论上这样处理应该是可行的呀，特征之间计算相似度去除了特征向量本身长度的影响。但是我这样处理后，loss降不下去。请问您有什么建议吗？或者说我这样处理本身就不对吗？

Hi, nice work! How did you determine the number of training iterations for each datasets? such as 800K for Things and 200K for Sintel. Is this the optimal number of iterations? How did you select and set them?

您好，我使用您提供的指令CUDA_VISIBLE_DEVICES=0 python main.py
--inference_dir demo/sintel_market_1
--output_path output/gmflow-norefine-sintel_market_1
--resume pretrained/gmflow_sintel-0c07dcb3.pth进行测试，sintel_market_1文件夹是您提供的图片，这个文件夹有三种图片，我删除了一张图片，只留下frame_0001.png和frame_0002.png。这个预测出来的光流，是frame_0001.png到frame_0002.png的光流吗？如果是这样的话，显示出的光流图根据颜色来看好像反了。

Congratulations on your new work!

I think the idea of flow propagation in GMFlow is similar to the flow field deformation in LiteFlowNet3. Both of us propagate flow predictions and measure feature self-similarity. However, LiteFlowNet3 was not cited in GMFlow.

Hi, Thanks for the great code! I find that there is GMFlow+ on the Sintel benchmark. It says GMFlow+ is a straightforward extention of GMFlow and achieves significant results. Could you give some details about GMFlow+ or the difference between GMFlow+ and GMFlow? Besides, I wonder whether the code of GMFlow+ will be released. Thanks!

First I must appreciate that you opened public code for your novel work.

I have a question about your score on the official MPISintel leaderboard.

According to an official paper published in CVPR2022, GMFlow's AEPE for Sintel clean and final test set was 1.74 and 2.90 but in the official leaderboard, GMFlow+'s AEPE is 1.028 and 2.367 for Sintel clean and final test set.

Can you tell me which extension added for GMFlow leads to such extreme performance improvement?

Code snippets: flow = self.feature_flow_attn(feature0, flow.detach(), local_window_attn=prop_radius > 0, local_window_radius=prop_radius)

I'm curious about the flow.detach(). What was your idea for that design? I guess multipath backpropagation affects optimization performance, but I don't know if it's right.

您好，感谢您慷慨地将此工作设置为开放获取。

我发现目前的版本似乎不支持直接对视频流进行推理。我不得不 video2images + Img2Video 后来获得视频流的光流结果。 如果能增加对视频流的实时推理将更好(这对GMFlow似乎是有难度的)。我在RTX3060上对651张图片进行了推理，耗时420.8246901035309秒。FlowNet2似乎能做到实时(从他们的演示视频上来看)。

这只是一个友好的建议，感谢将该工作开源。

大佬您好

我想向您请教一下

【1】您的训练流程是不是和train_gmflow.sh或train_gmflow_with_refine.sh一样：先在chairs上训练，然后在Things上微调，然后在sintel上微调，然后在kitti上微调。这4个数据集都需要跑一遍，而且是一环扣一环呢？

【2】这4个数据集都跑一遍（在4张V100上）需要多长时间呢？我目前的设备是5张3090

祝好！！！