Planar Surface Reconstruction From Sparse Views
Linyi Jin, Shengyi Qian, Andrew Owens, David F. Fouhey
University of Michigan
ICCV 2021 (Oral)
This repo contains code for our paper. Our model is implemented in Detectron2.
Given two RGB images with an unknown relationship, our system produces a single, coherent planar surface reconstruction of the scene in terms of 3D planes and relative camera poses.
We use a ResNet50-FPN to detect planes and predict probabilities of relative camera poses, and use a two-step optimization to generate a coherent planar reconstruction. (a) For each plane, we predict a segmentation mask, plane parameters, and an appearance feature. (b) Concurrently, we pass image features from the detection backbone through the attention layer and predict the camera transformation between views. (c) Our discrete optimization fuses the prediction of the separate heads to select the best camera pose and plane correspondence. (d) Finally, we use continuous optimization to update the camera and plane parameters.
Usage Instructions
- How to setup your environment?
- How to inference the code on a pair of images?
- How to process the dataset?
- How to train your model?
- How to evaluate your model?
Citation
If you find this code useful, please consider citing:
@inproceedings{jin2021planar,
title={Planar Surface Reconstruction from Sparse Views},
author={Linyi Jin and Shengyi Qian and Andrew Owens and David F. Fouhey},
booktitle = {ICCV},
year={2021}
}
Acknowledgment
We thank Dandan Shan, Mohamed El Banani, Nilesh Kulkarni, Richard Higgins for helpful discussions. Toyota Research Institute ("TRI") provided funds to assist the authors with their research but this article solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity.
37 Dec 4, 2022
36 Jan 3, 2023
111 Dec 29, 2022
249 Dec 28, 2022
151 Dec 26, 2022
305 Dec 21, 2022
85 Jan 4, 2023
521 Dec 30, 2022
14 Nov 28, 2022
59 Nov 25, 2022
23 Jan 4, 2023
47 Nov 18, 2022
264 Dec 23, 2022
1.4k Dec 30, 2022
302 Dec 14, 2022
127 Dec 31, 2022
125 Dec 2, 2022
48 Dec 15, 2022