Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

sunset

Last update: Jan 3, 2023

Related tags

Deep Learning PanoPlane360

Overview

HV-plane reconstruction from a single 360 image

Code for our paper in CVPR 2021: Indoor Panorama Planar 3D Reconstruction via Divide and Conquer (paper, video)

Pretrained models

Download our pretrained models from google drive or dropbox.

Inference on 360 datas

Please resize your images into 512 x 1024.
Follow the preprocessing step here to ensure Mahattan alignment of your 360 images.
Run our inference script. Examples:

python inference.py --pth ckpt/mp3d.pth --glob static/demo.png --outdir static/mp3d_model_results

To run on a batch of images, you can use --glob "AWESOME_360_IMAGES_DIR/*png"

Visualize the results

Here is the visulization example on a held-out data:

python vis_planes.py --img static/demo.png --h_planes static/mp3d_model_results/demo.h_planes.exr --v_planes static/mp3d_model_results/demo.v_planes.exr --mesh

To always visualize all the planes, add --mesh_show_back_face.

Citation

@inproceedings{SunHWSC21,
  author    = {Cheng Sun and
               Chi{-}Wei Hsiao and
               Ning{-}Hsu Wang and
               Min Sun and
               Hwann{-}Tzong Chen},
  title     = {Indoor Panorama Planar 3D Reconstruction via Divide and Conquer},
  booktitle = {CVPR},
  year      = {2021},
}

You might also like...

PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

Contrast to Divide: self-supervised pre-training for learning with noisy labels This is an official implementation of "Contrast to Divide: self-superv

55 Nov 23, 2022

N-Person-Check-Checker-Splitter - A calculator app use to divide checks

N-Person-Check-Checker-Splitter This is my from-scratch programmed calculator ap

2 Feb 15, 2022

Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV @ CVPR 2021.

MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

47 Nov 18, 2022

Planar Prior Assisted PatchMatch Multi-View Stereo

ACMP [News] The code for ACMH is released!!! [News] The code for ACMM is released!!! About This repository contains the code for the paper Planar Prio

127 Dec 31, 2022

This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection

Semantic SLAM This package can perform optimization of pose estimated from VO/VIO methods which tend to drift over time. It uses planar surfaces extra

125 Dec 2, 2022

PyTorch implementation of HDN(Homography Decomposition Networks) for planar object tracking

Homography Decomposition Networks for Planar Object Tracking This project is the offical PyTorch implementation of HDN(Homography Decomposition Networ

48 Dec 15, 2022

Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

NeurMips: Neural Mixture of Planar Experts for View Synthesis This is the official repo for PyTorch implementation of paper "NeurMips: Neural Mixture

101 Dec 13, 2022

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

274 Jan 5, 2023

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments Paper: arXiv (ICRA 2021) Video : https://youtu.be/CC

68 Jan 3, 2023

Comments

don't save v_planes.exr
Run our inference script we have to exr files v_planes and h_planes, but v_planes(segmentation) is not saved completely and will not open.

{'h_planes': array([[ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], [ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], [ 1.0249354, 1.0249354, 1.0249354, ..., 1.0249354, 1.0249354, 1.0249354], ..., [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685], [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685], [-0.9487685, -0.9487685, -0.9487685, ..., -0.9487685, -0.9487685, -0.9487685]], dtype=float32), 'v_planes': array([[[0., 0., 0.], [0., 0., 0.], [0., 0., 0.], ..., [0., 0., 0.], [0., 0., 0.], [0., 0., 0.]],

[[0., 0., 0.], [0., 0., 0.], [0., 0., 0.], ............................

predict v_planes is empty

Can u help ?
comeback to it this weekend
opened by KirillHiddleston 3
inference

File "inference.py", line 68, in []

"Could not find a format to write the specified file in %s mode" % modename ValueError: Could not find a format to write the specified file in single-image mode

on demo.png

opened by KirillHiddleston 1
Small holes when visualizing?

Hey there,

When running the visualization code I get small holes in the side of the walls

Do you have any idea why this may have occurred? Or do you apply some post processing to make the results cleaner?

Thanks in advance

opened by stefanklut 0

Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Related tags

Overview

HV-plane reconstruction from a single 360 image

Pretrained models

Inference on 360 datas

Visualize the results

Citation

You might also like...

PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

N-Person-Check-Checker-Splitter - A calculator app use to divide checks

Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV @ CVPR 2021.

Planar Prior Assisted PatchMatch Multi-View Stereo

This package is for running the semantic SLAM algorithm using extracted planar surfaces from the received detection

PyTorch implementation of HDN(Homography Decomposition Networks) for planar object tracking

Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Comments

don't save v_planes.exr

inference

Small holes when visualizing?

Owner

sunset

A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

[ICCV 2021 (oral)] Planar Surface Reconstruction from Sparse Views

Image-Stitching - Panorama composition using SIFT Features and a custom implementaion of RANSAC algorithm

Office source code of paper UniFuse: Unidirectional Fusion for 360$^\circ$ Panorama Depth Estimation

[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

A modified version of DeepMind's Alphafold2 to divide CPU part (MSA and template searching) and GPU part (prediction model)

Automatically download the cwru data set, and then divide it into training data set and test data set

Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

Utility tools for the "Divide and Remaster" dataset, introduced as part of the Cocktail Fork problem paper