Thank you for your excellent work. However, I have a doubt. According to the statement of your paper, the depth assumption of SRC image is used to obtain the similarity score by projection transformation to ref image. However, in Code, the depth assumption is set in the Ref Image coordinate system to obtain 3D points, which are transformed to the SRC image coordinate system and sampled from the SRC image. Finally, the similarity score is obtained by dot product with the REF image. This is actually helpful to get the depth of the ref image, instead of SRC image. I don't understand why you make the depth assumption in the Ref image coordinate system and then project it.

Thanks for sharing of code.

But I wonder why you did not test your method on 7Scenes or Tanks&Temple.

Besides, I tried to test your method on 7Scenes, but the results are pretty poor. Can you give me some suggestions?

My code is here: https://github.com/Yannnnnnnnnnnn/MVS2D.git at test_7scenes_long, as I use the split from Long.

My results is here:

7scenes:{'a1': 0.39494346839485406,
 'a2': 0.6459305167578225,
 'a3': 0.8022140821127047,
 'abs_diff': 0.5733749950019752,
 'abs_diff_median': 0.5094702287105953,
 'abs_rel': 0.32380217413691914,
 'log10': 0.17217029032900052,
 'rmse': 0.6894540686467114,
 'rmse_log': 0.47906266186605484,
 'sq_rel': 0.27828435772756005,
 'thre1': 0.23810647116173855,
 'thre3': 0.5398887457282459,
 'thre5': 0.8326691103244529}

hi I have some questions on convergence speed of MVS2D

the cost-volume based method (MVSNet, etc.) can see the outline of objects after 1 epoch or even several iters

however, MVS2D seems hard to convergence and the depthmap are very vague at the beginning.

does this due to the network design? Could you explain about this phenomenon?

There is a point in your repo that is confusing for newbies(for me), that the val set in DTU.py is not consistent with those in patchmatch_fusion.py

in DTU.py val it is

        data_set = [

            3, 5, 17, 21, 28, 35, 37, 38, 40, 43, 56, 59, 66, 67, 82, 86,
            106, 117
        ]

however in patchmatch_fusion.py / dtu_pyeval.py it is

scans = [

    1, 4, 9, 10, 11, 12, 13, 15, 23, 24, 29, 32, 33, 34, 48, 49, 62, 75,
    77, 110, 114, 118
]

hope you could fix it, or at least this post may be helpful for users who want to reproduce your great work！

Additionally, we use an asterisk sign ‘∗’ to denote an oracle version Bts∗, where we use the ground truth depth map to factor out the global scale.

Is it means that the mono depth network's prediction may not have the same scale as the depth map and Bts* is to estimate some parameter (e.g. mean and std in (depth-mean)/std) to adjust its scale according to the gt depth map?

Hi guys, Thanks for your great work Would you please let me know what is the following error about ? I did some search but I couldnt find the solution I would appresiate it if you could help thanks

**Traceback (most recent call last): File "demo.py", line 72, in outputs = model(imgs[0], imgs[1:], proj_mats[0], proj_mats[1:], inv_K_pool) File "/home/akarami/anaconda3/envs/mvs2d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, kwargs) File "/home/akarami/mvs2d/networks/mvs2d.py", line 311, in forward src_imgs, File "/home/akarami/mvs2d/networks/mvs2d.py", line 230, in epipolar_fusion k, proj_mask, grid = homo_warping(k, src_proj, ref_proj,depth_values) File "/home/akarami/mvs2d/networks/module.py", line 72, in homo_warping proj = torch.matmul(src_proj, torch.inverse(ref_proj)) RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

In the 'train.py' code, the input color images are resized and transposed by (1,2,0).

Why did you transposed the original input images when you save the "data" including original images if opt.mode=='test' and opt.save_prediction==True. ?

As you mentioned at the MVS2D paper,

ScanNet dataset sampled from original set by 86324 triple images for training and 666 triple images for testing.

I wanna know the criterion of frame sampling.

Hi, there is a bug in your code that cause: 'tuple' object is not callable training process is ok to run but always showing 'tuple' object is not callable

the log is:

Training 'tuple' object is not callable 'tuple' object is not callable 'tuple' object is not callable /mypath/anaconda3/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:129: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate warnings.warn("Detected call of lr_scheduler.step() before optimizer.step(). " /mypath/anaconda3/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:154: UserWarning: The epoch parameter in scheduler.step() was not necessary and is being deprecated where possible. Please use scheduler.step() to step the scheduler. During the deprecation, if epoch is different from None, the closed form is used instead of the new chainable form, where available. Please open an issue if you are unable to replicate your use case: https://github.com/pytorch/pytorch/issues/new/choose. warnings.warn(EPOCH_DEPRECATION_WARNING, UserWarning) 'tuple' object is not callable /mypath/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.) return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) 'tuple' object is not callable 'tuple' object is not callable 'tuple' object is not callable