VRCNet: Variational Relational Point Completion Network
This repository contains the PyTorch implementation of the paper:
Variational Relational Point Completion Network, CVPR 2021 (Oral)
In CVPR 2021
Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds. One path consumes complete point clouds for reconstruction by learning a point VAE. The other path generates complete shapes for partial point clouds, whose embedded distribution is guided by distribution obtained from the reconstruction path during training. 2) Relational Enhancement. Specifically, we carefully design point selfattention kernel and point selective kernel module to exploit relational point features, which refines local shape details conditioned on the coarse completion. In addition, we contribute a multi-view partial point cloud dataset (MVP dataset) containing over 100,000 high-quality scans, which renders partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model. Extensive experiments demonstrate that VRCNet outperforms state-of-theart methods on all standard point cloud completion benchmarks. Notably, VRCNet shows great generalizability and robustness on real-world point cloud scans.
VRCNet architecture overview:
Our proposed point cloud learning modules:
Point Cloud Completion Benchmark
Moreover, this repository introduces an integrated Point Cloud Completion Benchmark implemented in Python 3.5, PyTorch 1.2 and CUDA 10.0. Supported algorithms: PCN, Topnet, MSN, Cascade, ECG and our VRCNet.
Installation
- Install dependencies:
- h5py 2.10.0
- matplotlib 3.0.3
- munch 2.5.0
- open3d 0.9.0
- PyTorch 1.2.0
- PyYAML 5.3.1
-
Download corresponding dataset (e.g. MVP dataset)
-
Compile PyTorch 3rd-party modules (ChamferDistancePytorch, emd, expansion_penalty, MDS, Pointnet2.PyTorch)
MVP Dataset
Please download our MVP Dataset to the folder data.
Usage
- To train a model: run
python train.py -c *.yaml, e.g.python train.py -c pcn.yaml - To test a model: run
python test.py -c *.yaml, e.g.python test.py -c pcn.yaml - Config for each algorithm can be found in
cfgs/. run_train.shandrun_test.share provided for SLURM users.
Citation
If you find our code useful, please cite our paper:
@article{pan2021vrcnet,
title={Variational Relational Point Completion Network},
author={Pan, Liang and Chen, Xinyi and Cai, Zhongang and Zhang, Junzhe and Zhao, Haiyu and Yi, Shuai and Liu, Ziwei},
journal={arXiv preprint arXiv:2104.10154},
year={2021}
}
License
Our code is released under MIT License.
Acknowledgement
We include the following PyTorch 3rd-party libraries:
[1] ChamferDistancePytorch
[2] emd, expansion_penalty, MDS
[3] Pointnet2.PyTorch
We include the following algorithms:
[1] PCN
[2] MSN
[3] Topnet
[4] Cascade
[5] ECG
[6] VRCNet
119 Jan 2, 2023
800 Dec 5, 2022
317 Dec 26, 2022
259 Dec 25, 2022
43 Dec 5, 2022
264 Dec 23, 2022
197 Dec 9, 2022
135 Dec 26, 2022
76 Dec 13, 2022
100 Dec 22, 2022
48 Dec 23, 2022
93 Dec 21, 2022
22 Jan 4, 2023
37 Dec 22, 2022
48 Nov 23, 2022
40 Dec 30, 2022
9 Dec 7, 2022
66 Nov 30, 2022
722 Dec 27, 2022