Implementation of Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou

Last update: Dec 29, 2022

Related tags

Deep Learning PVD

Overview

Shape Generation and Completion Through Point-Voxel Diffusion

Project | Paper

Implementation of Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou, Yilun Du, Jiajun Wu

Requirements:

Make sure the following environments are installed.

python==3.6
pytorch==1.4.0
torchvision==0.5.0
cudatoolkit==10.1
matplotlib==2.2.5
tqdm==4.32.1
open3d==0.9.0
trimesh=3.7.12
scipy==1.5.1

Install PyTorchEMD by

cd metrics/PyTorchEMD
python setup.py install
cp build/**/emd_cuda.cpython-36m-x86_64-linux-gnu.so .

The code was tested on Unbuntu with Titan RTX.

Data

For generation, we use ShapeNet point cloud, which can be downloaded here.

For completion, we use ShapeNet rendering provided by GenRe. We provide script convert_cam_params.py to process the provided data.

For training the model on shape completion, we need camera parameters for each view which are not directly available. To obtain these, simply run

$ python convert_cam_params.py --dataroot DATA_DIR --mitsuba_xml_root XML_DIR

which will create ..._cam_params.npz in each provided data folder for each view.

Pretrained models

Pretrained models can be downloaded here.

Training:

$ python train_generation.py --category car|chair|airplane

Please refer to the python file for optimal training parameters.

Testing:

$ python train_generation.py --category car|chair|airplane --model MODEL_PATH

Results

Some generation and completion results are as follows.

Multimodal completion on a ShapeNet chair.

Multimodal completion on PartNet.

Multimodal completion on two Redwood 3DScan chairs.

Reference

@inproceedings{Zhou_2021_ICCV,
    author    = {Zhou, Linqi and Du, Yilun and Wu, Jiajun},
    title     = {3D Shape Generation and Completion Through Point-Voxel Diffusion},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {5826-5835}
}

Acknowledgement

For any questions related to codes and experiment setting, please contact Linqi Zhou and Yilun Du.

You might also like...

MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration [NEWS] 2021-07-12 [NEW 🎉 ] The submission on Codalab starts! 2021-07-1

93 Dec 21, 2022

[CVPR 2021 Oral] Variational Relational Point Completion Network

VRCNet: Variational Relational Point Completion Network This repository contains the PyTorch implementation of the paper: Variational Relational Point

121 Dec 12, 2022

[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

317 Dec 26, 2022

Official page of Struct-MDC (RA-L'22 with IROS'22 option); Depth completion from Visual-SLAM using point & line features

Struct-MDC (click the above buttons for redirection!) Official page of "Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural R

37 Dec 22, 2022

Implementation of GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation (ICLR 2022).

GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [OpenReview] [arXiv] [Code] The official implementation of GeoDiff: A Geome

155 Dec 26, 2022

Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng Internati

115 Jan 4, 2023

Voxel Transformer for 3D object detection

Voxel Transformer This is a reproduced repo of Voxel Transformer for 3D object detection. The code is mainly based on OpenPCDet. Introduction We provi

173 Dec 25, 2022

Python code to fuse multiple RGB-D images into a TSDF voxel volume.

Volumetric TSDF Fusion of RGB-D Images in Python This is a lightweight python script that fuses multiple registered color and depth images into a proj

845 Jan 3, 2023

for taichi voxel-challange event

Taichi Voxel Challenge Figure: result of python3 example6.py. Please replace the image above (demo.jpg) with yours, so that other people can immediate

20 Nov 26, 2022

Comments

How to install the module:"_pvcnn_backend"

No module named '_pvcnn_backend' File "/home//MyProjects/PVD/modules/functional/backend.py", line 22, in 'bindings.cpp', File "/home//MyProjects/PVD/modules/functional/ball_query.py", line 3, in from modules.functional.backend import _backend File "/home//MyProjects/PVD/modules/functional/init.py", line 1, in from modules.functional.ball_query import ball_query File "/home//MyProjects/PVD/modules/ball_query.py", line 4, in import modules.functional as F File "/home/**/MyProjects/PVD/modules/init.py", line 1, in from modules.ball_query import BallQuery File "/home/MyProjects/PVD/model/pvcnn_generation.py", line 6, in from modules import SharedMLP, PVConv, PointNetSAModule, PointNetAModule, PointNetFPModule, Attention, Swish File "/home//MyProjects/PVD/train_generation.py", line 11, in from model.pvcnn_generation import PVCNN2Base

opened by RavenScroft863 1
why need the noise init

thanks for you excellent work. I have a question here, for the cases of image generation using DDPM, no noise init is provided. so why we need the noise init in the code ?

opened by winnechan 1
Questions about metrics

Hi! Thanks for your wonderful work.

I want to ask a question about the metrics. I read your code in test_generation.py and find that the metrics are calculated between generated sample and test sample read from dataset. In your diffusion model, generated samples are random generated and it may differ in each running result. So when calculating the Chamfer Distance( calculate the distance between generated samples and test samples), it may cause a doubt result. I want to know how to guarantee a reliable result.

Best.

opened by yufeng9819 1
EMD doesn't match with PointFlow's

Hi,

I used your EMD code and PointFlow's to compute and find the two do not match. Yours is approximately 10 times less than PointFlow's. Could you please check this?

Airplane: Yours {'1-NN-CD-acc': 0.7259259223937988, '1-NN-CD-acc_f': 0.6370370388031006, '1-NN-CD-acc_t': 0.8148148059844971, '1-NN-EMD-acc': 0.614814817905426, '1-NN-EMD-acc_f': 0.5629629492759705, '1-NN-EMD-acc_t': 0.6666666865348816, 'lgan_cov-CD': 0.4962962865829468, 'lgan_cov-EMD': 0.47654321789741516, 'lgan_mmd-CD': 0.00022858072770759463, 'lgan_mmd-EMD': 0.0035688027273863554, 'lgan_mmd_smp-CD': 0.0007089844439178705, 'lgan_mmd_smp-EMD': 0.006884903181344271} 'JSD: 0.04573863398532296'

PointFlow's {'1-NN-CD-acc': 0.7259259223937988, '1-NN-CD-acc_f': 0.6370370388031006, '1-NN-CD-acc_t': 0.8148148059844971, '1-NN-EMD-acc': 0.6740740537643433, '1-NN-EMD-acc_f': 0.6320987939834595, '1-NN-EMD-acc_t': 0.7160493731498718, 'lgan_cov-CD': 0.4962962865829468, 'lgan_cov-EMD': 0.5185185074806213, 'lgan_mmd-CD': 0.00022858072770759463, 'lgan_mmd-EMD': 0.03101278468966484, 'lgan_mmd_smp-CD': 0.0007089844439178705, 'lgan_mmd_smp-EMD': 0.04470111057162285} 'JSD: 0.04573863398532296'

opened by minzhang-1 1

Implementation of Shape Generation and Completion Through Point-Voxel Diffusion

Related tags

Overview

Shape Generation and Completion Through Point-Voxel Diffusion

Requirements:

Data

Pretrained models

Training:

Testing:

Results

Reference

Acknowledgement

You might also like...

MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

[CVPR 2021 Oral] Variational Relational Point Completion Network

[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

Official page of Struct-MDC (RA-L'22 with IROS'22 option); Depth completion from Visual-SLAM using point & line features

Implementation of GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation (ICLR 2022).

Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

Voxel Transformer for 3D object detection

Python code to fuse multiple RGB-D images into a TSDF voxel volume.

for taichi voxel-challange event

Comments

How to install the module:"_pvcnn_backend"

why need the noise init

Questions about metrics

EMD doesn't match with PointFlow's

Owner

Linqi Zhou

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

[ICCV 2021 Oral] SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer

Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

Minimal diffusion models - Minimal code and simple experiments to play with Denoising Diffusion Probabilistic Models (DDPMs)

PyTorch implementation for View-Guided Point Cloud Completion

This repo is a PyTorch implementation for Paper "Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds"

Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

PyTorch implementation of "LayoutTransformer: Layout Generation and Completion with Self-attention"