[ICCV'21] PlaneTR: Structure-Guided Transformers for 3D Plane Recovery

Last update: Nov 30, 2022

Overview

PlaneTR: Structure-Guided Transformers for 3D Plane Recovery

This is the official implementation of our ICCV 2021 paper

News

There maybe some bugs in the current public code and I am trying my best to solve them.

Contact me if you have any question.

TODO

Supplement 2D/3D visualization code.

Getting Started

Clone the repository:

git clone https://github.com/IceTTTb/PlaneTR3D.git

We use Python 3.6 and PyTorch 1.6.0 in our implementation, please install dependencies:

conda create -n planeTR python=3.6
conda activate planeTR
conda install pytorch=1.6.0 torchvision=0.7.0 torchaudio cudatoolkit=10.2 -c pytorch
pip install -r requirements.txt

Data Preparation

We train and test our network on the plane dataset created by PlaneNet. We follow PlaneAE to convert the .tfrecords to .npz files. Please refer to PlaneAE for more details.

We generate line segments using the state-of-the-art line segment detection algorithm HAWP with their pretrained model. The processed line segments data we used can be downloaded here.

The structure of the data folder should be

plane_data/
  --train/*.npz
  --train_img/*
  --val/*.npz
  --val_img/*
  --train.txt
  --val.txt

Training

Download the pretrained model of HRNet and place it under the 'ckpts/' folder.

Change the 'root_dir' in config files to the path where you save the data.

Run the following command to train our network on one GPU:

CUDA_VISIBLE_DEVICES=0 python train_planeTR.py

Run the following command to train our network on multiple GPUs:

CUDA_VISIBLE_DEVICES=0,1,2 python -m torch.distributed.launch --nproc_per_node=3 --master_port 295025 train_planeTR.py

Evaluation

Download the pretrained model here and place it under the 'ckpts/' folder.

Change the 'resume_dir' in 'config_planeTR_eval.yaml' to the path where you save the weight file.

Change the 'root_dir' in config files to the path where you save the data.

Run the following command to evaluate the performance:

CUDA_VISIBLE_DEVICES=0 python eval_planeTR.py

Citations

If you find our work useful in your research, please consider citing:

@inproceedings{tan2021planeTR,
title={PlaneTR: Structure-Guided Transformers for 3D Plane Recovery},
author={Tan, Bin and Xue, Nan and Bai, Song and Wu, Tianfu and Xia, Gui-Song},
booktitle = {International Conference on Computer Vision},
year={2021}
}

Contact

[email protected]

https://xuenan.net/

Acknowledgements

We thank the authors of PlaneAE, PlaneRCNN, interplane and DETR. Our implementation is heavily built upon their codes.

You might also like...

DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021)

Evaluation, Training, Demo, and Inference of DeFMO DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021) Denys Rozumnyi, Martin R. O

139 Dec 26, 2022

HandTailor: Towards High-Precision Monocular 3D Hand Recovery

HandTailor This repository is the implementation code and model of the paper "HandTailor: Towards High-Precision Monocular 3D Hand Recovery" (arXiv) G

113 Jan 6, 2023

Learning Energy-Based Models by Diffusion Recovery Likelihood

Learning Energy-Based Models by Diffusion Recovery Likelihood Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, Diederik P. Kingma Paper: https://arxiv.o

41 Nov 22, 2022

Unsupervised 3D Human Mesh Recovery from Noisy Point Clouds

Unsupervised 3D Human Mesh Recovery from Noisy Point Clouds Xinxin Zuo, Sen Wang, Minglun Gong, Li Cheng Prerequisites We have tested the code on Ubun

41 Dec 12, 2022

(ICCV 2021) ProHMR - Probabilistic Modeling for Human Mesh Recovery

ProHMR - Probabilistic Modeling for Human Mesh Recovery Code repository for the paper: Probabilistic Modeling for Human Mesh Recovery Nikos Kolotouros

209 Dec 13, 2022

Multivariate Time Series Forecasting with efficient Transformers. Code for the paper "Long-Range Transformers for Dynamic Spatiotemporal Forecasting."

Spacetimeformer Multivariate Forecasting This repository contains the code for the paper, "Long-Range Transformers for Dynamic Spatiotemporal Forecast

440 Jan 2, 2023

Comments

Looking for train.txt and val.txt

Congratulations on great work ! I am just looking for two files named train.txt and val.txt but failed. Could you please tell me where i can get them or the details of them? Thanks a lot.

opened by yq-Liang 2
How to get the planes_scannet_train.tfrecords?

Hi,

Thank you for such a great job!

I have troubles when downloading the planes_scannet_train.tfrecords from the link provided by PlaneNet. I just could not download the data. Another box link does not work (the file has been removed). So would you mind sharing your downloaded .tfrecords or preprocessed .npz files? Thanks a lot!

opened by ArcherFMY 2
When loss_layer_num>1, why just use one layer for the auxiliary loss?

Thanks for your good works! loss_layer_num is set as 3 in the config. But only layer[1] and layer[-1] are optimized? https://github.com/IceTTTb/PlaneTR3D/blob/09b6df2e339bbadeb254af5c893174df0896353b/models/planeTR_HRNet.py#L347-L353

opened by ewrfcas 2