idn-solver
This repository contains the code release of our ICCV 2021 paper:
A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo
Wang Zhao*, Shaohui Liu*, Yi Wei, Hengkai Guo, Yong-Jin Liu
Installation
We recommend to use conda to setup a specified environment. Run
conda env create -f environment.yml
Test on a sequence
First download the pretrained model from here and put it under ./pretrain/ folder.
Prepare the sequence data with color images, camera poses (4x4 cam2world transformation) and intrinsics. The sequence data structure should be like:
sequence_name
| color
| 00000.jpg
| pose
| 00000.txt
| K.txt
Run the following command to get the outputs:
python infer_folder.py --seq_dir /path/to/the/sequence/data --output_dir /path/to/save/outputs --config ./configs/test_folder.yaml
Tune the "reference gap" parameter to make sure there are sufficient overlaps and camera translations within an image pair. For ScanNet-like sequence, we recommend to use reference_gap of 20.
Test on ScanNet
Prepare ScanNet test split data
Download the ScanNet test split data from the official site and pre-process the data using:
python ./data/preprocess.py --data_dir /path/to/scannet/test/split/ --output_dir /path/to/save/pre-processed/scannet/test/data
This includes 1. resize the color images to 480x640 resolution 2. sample the data with interval of 20
Run evaluation
python eval_scannet.py --data_dir /path/to/processed/scannet/test/split/ --config ./configs/test_scannet.yaml
Train
Prepare ScanNet training data
We use the pre-processed ScanNet data from NAS, you could download the data using this link. The data structure is like:
scannet
| scannet_nas
| train
| scene0000_00
| color
| 0000.jpg
| pose
| 0000.txt
| depth
| 0000.npy
| intrinsic
| normal
| 0000_normal.npy
| val
| scans_test_sample (preprocessed ScanNet test split)
Run training
Modify the "dataset_path" variable with yours in the config yaml.
The network is trained with a two-stage strategy. The whole training process takes ~6 days with 4 Nvidia V100 GPUs.
python train.py ./configs/scannet_stage1.yaml
python train.py ./configs/scannet_stage2.yaml
Citation
If you find our work useful in your research, please consider citing:
@InProceedings{Zhao_2021_ICCV,
author = {Zhao, Wang and Liu, Shaohui and Wei, Yi and Guo, Hengkai and Liu, Yong-Jin},
title = {A Confidence-Based Iterative Solver of Depths and Surface Normals for Deep Multi-View Stereo},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {6168-6177}
}
Acknowledgement
This project heavily relies codes from NAS and we thank the authors for releasing their code.
We also thank Xiaoxiao Long for kindly helping with ScanNet evaluations.
118 Jan 4, 2023
66 Jan 4, 2023
53 Dec 24, 2022
287 Jan 6, 2023
72 Nov 27, 2022
14 Nov 28, 2022
26 Nov 24, 2022
328 Jan 9, 2023
127 Jan 4, 2023
4.7k Jan 7, 2023
205 Nov 9, 2022
369 Dec 24, 2022
11 Feb 8, 2022
1.1k Jan 4, 2023
0 May 22, 2022
4 Feb 5, 2022
111 Dec 29, 2022
36 Dec 2, 2022