the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

Related tags

Deep Learning BGNet
Overview

BGNet

This repository contains the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

alt text

Environment

  1. Python 3.6.*
  2. CUDA 10.1
  3. PyTorch 1.7.1
  4. TorchVision 0.8.2

Dataset

To evaluate/train our BGNet network, you will need to download the required datasets.

Pretrained model

We provide seven pretrained model under the folder models .

Evaluation

We provided a script to get the kitti benchmark result,check predict.sh for an example usage.

Prediction

We support predicting on any rectified stereo pairs. predict_sample.py provides an example usage.

Acknowledgements

Part of the code is adopted from the previous works: DeepPruner, GwcNet, GANet and AANet. We thank the original authors for their contributions.

Citing

If you find this code useful, please consider to cite our work.

@inproceedings{xu2021bilateral,
  title={Bilateral Grid Learning for Stereo Matching Networks},
  author={Xu, Bin and Xu, Yuhua and Yang, Xiaoli and Jia, Wei and Guo, Yulan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={1--10},
  year={2021}
}
Comments
  • maxdisp

    maxdisp

    Hi, in the paper it's written that the maximal disparity value is set to 192 during evaluation. However, looking in your code, the maxdisp argument into build_gwc_volume() is set to 25 (e.g. in BGNet_Plus). Should this be 192, or did I understand the paper/code wrongly?

    opened by jwpleow 0
  • Training details

    Training details

    Hi，thank you for your greate work! I'm so interested in BGNet+, and trained it from scratch using Sceneflow dataset like your paper said. I have got EPE=1.17, but when I inference my own images using my trained model, the disparity quality is worse than using your pretrained model. so can you tell me more training details about your model? thanks very much ,and hope to receive your reply. Thanks!

    opened by dongda118 0
  • 关于您所发表的CVPR 2021论文《Bilateral Grid Learning for Stereo Matching Network》实验结果的疑问，希望能得到您的解答

    关于您所发表的CVPR 2021论文《Bilateral Grid Learning for Stereo Matching Network》实验结果的疑问，希望能得到您的解答

    首先，感谢您将这个令人印象深刻的工作开源。在研读CVPR 2021论文《Bilateral Grid Learning for Stereo Matching Network》后，我使用您在Github上公布的预训练模型Sceneflow-BGNet.pth和Sceneflow-BGNet-Plus.pth在SceneFlow测试集上进行测试，期望能够复现论文Table 1中CUBG的EPE指标（如下图所示）。

    image

    但是我只能获得比EPE=1.17更差的指标，我获得的实际指标如下： （1）Sceneflow-BGNet.pth，模型参数量为2974770: epe: 1.301 d1: 0.0535 thres1: 0.1452 thres2: 0.0822 thres3: 0.0616 （2）Sceneflow-BGNet-Plus.pth，模型参数量为5315811: epe: 1.167 d1: 0.0502 thres1: 0.1343 thres2: 0.0785 thres3: 0.0587。

    所以，我的问题是：

    1. 按照论文中的上下文推断，Table 1中的CUBG模型应该指的是Sceneflow-BGNet.pth，然而使用您所公布的预训练模型Sceneflow-BGNet.pth，我只能获得EPE=1.301。请问，在SceneFlow finalpass 测试集上使用预训练模型Sceneflow-BGNet.pth时，对原始输入图像需要做怎样的预处理，才能获得论文Table 1中的CUBG EPE=1.17的指标？
    2. 我使用Sceneflow-BGNet-Plus.pth预训练模型，在SceneFlow数据集上可以获得EPE=1.167，与Table 1中的1.17基本一致。请问，论文Table 1中的EPE=1.17这个数值，具体是Sceneflow-BGNet.pth的结果，还是Sceneflow-BGNet-Plus.pth的结果？

    非常感谢。期待您的回复。

    opened by wuzhongwulidong 0
  • Error when using your pretrained Sceneflow-BGNet.pth Model !

    Error when using your pretrained Sceneflow-BGNet.pth Model !

    Great work! However there is a mistake when using your pretrained Sceneflow-BGNet.pth Model ! As when all know, SceneFlow dataset consists of color images of 3 channels ,but in your code: self.feature_extraction = feature_extraction(image_planes) self.firstconv = nn.Sequential(convbn_relu(1, 32, 3, 2, 1, 1), that is to say your model only accept one channel images. Also, I find that your pretrained Sceneflow-BGNet.pth just exactly accept one channel images. So how can I reproduce the EPE result of CUBG on SceneFlow using your pretrained model, as your paper report CUBG EPE=1.17 in Table 1?

    opened by wuzhongwulidong 1
  • What's wa and wb？

    What's wa and wb？

    Thank you for your wonderful work! Could you explain that what are the learnable parameters wa and wb? It seems like they are not mentioned in the original paper? https://github.com/3DCVdeveloper/BGNet/blob/19b9c1bc17ebf653b344e52470ae6f85953c17a9/models/bgnet_plus.py#L109

    opened by Sarah20187 1
Owner
3DCV developer
3D computer vision developer community is a professional technological communication forum for developers.
3DCV developer
GitHub
Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021) This repository is for BAAF-Net introduce

null 90 Dec 29, 2022
Stacked Recurrent Hourglass Network for Stereo Matching

SRH-Net: Stacked Recurrent Hourglass Introduction This repository is supplementary material of our RA-L submission, which helps reviewers to understan

null 28 Jan 3, 2023
Lightweight stereo matching network based on MobileNetV1 and MobileNetV2

MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching

Cognitive Systems Research Group 139 Nov 30, 2022
Pytorch reimplementation of PSM-Net: "Pyramid Stereo Matching Network"

This is a Pytorch Lightning version PSMNet which is based on JiaRenChang/PSMNet. use python main.py to start training. PSM-Net Pytorch reimplementatio

XIAOTIAN LIU 1 Nov 25, 2021
Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch .

PyTorch-High-Res-Stereo-Depth-Estimation Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch. Stereo dep

Ibai Gorordo 26 Nov 24, 2022
Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"

Dataset Distillation by Matching Training Trajectories Project Page | Paper This repo contains code for training expert trajectories and distilling sy

George Cazenavette 256 Jan 5, 2023
The official implementation code of "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction."

PlantStereo This is the official implementation code for the paper "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction".

Wang Qingyu 14 Nov 28, 2022
Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"

SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo Thomas Kollar, Michael Laskey, Kevin Stone, Brijen Thananjeyan

null 68 Dec 14, 2022
Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.

C2-Matching (CVPR2021) This repository contains the implementation of the following paper: Robust Reference-based Super-Resolution via C2-Matching Yum

Yuming Jiang 151 Dec 26, 2022
CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching（CVPR2021）

CFNet(CVPR 2021) This is the implementation of the paper CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching, CVPR 2021, Zhelun Shen, Yuch

null 106 Dec 28, 2022
✨✨✨An awesome open source toolbox for stereo matching.

OpenStereo This is an awesome open source toolbox for stereo matching. Supported Methods: BM SGM(T-PAMI'07) GCNet(ICCV'17) PSMNet(CVPR'18) StereoNet(E

Wang Qingyu 6 Nov 4, 2022
Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks

Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks Contributions A novel pairwise feature LSP to extract structural

null 31 Dec 6, 2022
Official code for the paper: Deep Graph Matching under Quadratic Constraint (CVPR 2021)

QC-DGM This is the official PyTorch implementation and models for our CVPR 2021 paper: Deep Graph Matching under Quadratic Constraint. It also contain

Quankai Gao 55 Nov 14, 2022
[CVPR 2022] CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation

CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation Prerequisite Please create and activate the following conda envrionment. To r

Qin Wang 87 Jan 8, 2023
Official code for On Path Integration of Grid Cells: Group Representation and Isotropic Scaling (NeurIPS 2021)

On Path Integration of Grid Cells: Group Representation and Isotropic Scaling This repo contains the official implementation for the paper On Path Int

Ruiqi Gao 39 Nov 10, 2022
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV @ CVPR 2021.

MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
Asymmetric Bilateral Motion Estimation for Video Frame Interpolation, ICCV2021

ABME (ICCV2021) Junheum Park, Chul Lee, and Chang-Su Kim Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolati

Junheum Park 86 Dec 28, 2022
BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis

Bilateral Denoising Diffusion Models (BDDMs) This is the official PyTorch implementation of the following paper: BDDM: BILATERAL DENOISING DIFFUSION M

null 172 Dec 23, 2022
Code for "LoFTR: Detector-Free Local Feature Matching with Transformers", CVPR 2021

LoFTR: Detector-Free Local Feature Matching with Transformers Project Page | Paper LoFTR: Detector-Free Local Feature Matching with Transformers Jiami

ZJU3DV 1.4k Jan 4, 2023