Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Overview

HV-plane reconstruction from a single 360 image

Code for our paper in CVPR 2021: Indoor Panorama Planar 3D Reconstruction via Divide and Conquer (paper, video)

teaser

Pretrained models

Download our pretrained models from google drive or dropbox.

Inference on 360 datas

  1. Please resize your images into 512 x 1024.
  2. Follow the preprocessing step here to ensure Mahattan alignment of your 360 images.
  3. Run our inference script. Examples:
python inference.py --pth ckpt/mp3d.pth --glob static/demo.png --outdir static/mp3d_model_results

To run on a batch of images, you can use --glob "AWESOME_360_IMAGES_DIR/*png"

Visualize the results

Here is the visulization example on a held-out data:

python vis_planes.py --img static/demo.png --h_planes static/mp3d_model_results/demo.h_planes.exr --v_planes static/mp3d_model_results/demo.v_planes.exr --mesh

To always visualize all the planes, add --mesh_show_back_face.

Citation

@inproceedings{SunHWSC21,
  author    = {Cheng Sun and
               Chi{-}Wei Hsiao and
               Ning{-}Hsu Wang and
               Min Sun and
               Hwann{-}Tzong Chen},
  title     = {Indoor Panorama Planar 3D Reconstruction via Divide and Conquer},
  booktitle = {CVPR},
  year      = {2021},
}
You might also like...
PyTorch implementation of
PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

Contrast to Divide: self-supervised pre-training for learning with noisy labels This is an official implementation of "Contrast to Divide: self-superv

N-Person-Check-Checker-Splitter - A calculator app use to divide checks

N-Person-Check-Checker-Splitter This is my from-scratch programmed calculator ap

Python and C++ implementation of
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV @ CVPR 2021.

MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Planar Prior Assisted PatchMatch Multi-View Stereo

ACMP [News] The code for ACMH is released!!! [News] The code for ACMM is released!!! About This repository contains the code for the paper Planar Prio

This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection
This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection

Semantic SLAM This package can perform optimization of pose estimated from VO/VIO methods which tend to drift over time. It uses planar surfaces extra

PyTorch implementation of HDN(Homography Decomposition Networks) for planar object tracking
PyTorch implementation of HDN(Homography Decomposition Networks) for planar object tracking

Homography Decomposition Networks for Planar Object Tracking This project is the offical PyTorch implementation of HDN(Homography Decomposition Networ

Pytorch implementation of paper:
Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

NeurMips: Neural Mixture of Planar Experts for View Synthesis This is the official repo for PyTorch implementation of paper "NeurMips: Neural Mixture

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments
Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments Paper: arXiv (ICRA 2021) Video : https://youtu.be/CC

Comments
  • don't save v_planes.exr

    don't save v_planes.exr

    Run our inference script we have to exr files v_planes and h_planes, but v_planes(segmentation) is not saved completely and will not open.

    {'h_planes': array([[ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], [ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], [ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], ..., [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685], [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685], [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685]], dtype=float32), 'v_planes': array([[[0., 0., 0.], [0., 0., 0.], [0., 0., 0.], ..., [0., 0., 0.], [0., 0., 0.], [0., 0., 0.]],

       [[0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        ............................
    

    predict v_planes is empty

    Can u help ?

    comeback to it this weekend 
    opened by KirillHiddleston 3
  •   inference

    inference

    File "inference.py", line 68, in []

    "Could not find a format to write the specified file in %s mode" % modename ValueError: Could not find a format to write the specified file in single-image mode

    on demo.png

    opened by KirillHiddleston 1
  • Small holes when visualizing?

    Small holes when visualizing?

    Hey there,

    When running the visualization code I get small holes in the side of the walls Screenshot from 2021-10-25 14-00-57

    Do you have any idea why this may have occurred? Or do you apply some post processing to make the results cleaner?

    Thanks in advance

    opened by stefanklut 0
Owner
sunset
CS Lover
sunset
A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

ManhattanSLAM Authors: Raza Yunus, Yanyan Li and Federico Tombari ManhattanSLAM is a real-time SLAM library for RGB-D cameras that computes the camera

null 117 Dec 28, 2022
git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction Code for the ECCV 2020 paper by Yiming Qian and Yasutaka Furukawa Getting

null 37 Dec 4, 2022
[ICCV 2021 (oral)] Planar Surface Reconstruction from Sparse Views

Planar Surface Reconstruction From Sparse Views Linyi Jin, Shengyi Qian, Andrew Owens, David F. Fouhey University of Michigan ICCV 2021 (Oral) This re

Linyi Jin 89 Jan 5, 2023
Image-Stitching - Panorama composition using SIFT Features and a custom implementaion of RANSAC algorithm

About The Project Panorama composition using SIFT Features and a custom implementaion of RANSAC algorithm (Random Sample Consensus). Author: Andreas P

Andreas Panayiotou 3 Jan 3, 2023
Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

UniFuse (RAL+ICRA2021) Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation, arXiv, Demo Preparation I

Alibaba 47 Dec 26, 2022
[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

Hou zhijian 23 Dec 26, 2022
A modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (prediction model)

ParallelFold Author: Bozitao Zhong This is a modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (p

Bozitao Zhong 77 Dec 22, 2022
Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集,然后分训练数据集和测试数据集

null 6 Jun 27, 2022
Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

Demonstration of OpenVINO techniques - Model-division and a simplest-way to support custom layers Description: Model Optimizer in Intel(r) OpenVINO(tm

Yasunori Shimura 12 Nov 9, 2022
Utility tools for the "Divide and Remaster" dataset, introduced as part of the Cocktail Fork problem paper

Divide and Remaster Utility Tools Utility tools for the "Divide and Remaster" dataset, introduced as part of the Cocktail Fork problem paper The DnR d

Darius Petermann 46 Dec 11, 2022