MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

Overview

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration

[NEWS]

[Timeline]

  • 2021-07-12   Submission start date
  • 2021-09-12   Public submission deadline
  • 2021-09-19   Private submission deadline
  • 2021-10-04   Technical report deadline
  • 2021-10-17   Awards at ICCV2021 Workshop

[MVP Benchmark]

Overview

This repository introduces the MVP Benchmark for partial point cloud COMPLETION and REGISTRATION, and it also includes following recent methods:

This repository is implemented in Python 3.7, PyTorch 1.5.0, CUDA 10.1 and gcc > 5.

Installation

Install Anaconda, and then use the following command:

git clone --depth=1 https://github.com/paul007pl/MVP_Benchmark.git
cd MVP_Benchmark; source setup.sh;

If your connection to conda and pip is unstable, it is recommended to manually follow the setup steps in setup.sh.

MVP Dataset

Download corresponding dataset:

Usage

For both completion and registration:

  • cd completion or cd registration

  • To train a model: run python train.py -c ./cfgs/*.yaml, e.g. python train.py -c ./cfgs/pcn.yaml

  • To test a model: run python test.py -c ./cfgs/*.yaml, e.g. python test.py -c ./cfgs/pcn.yaml

  • Config for each algorithm can be found in cfgs/.

  • run_train.sh and run_test.sh are provided for SLURM users.

  • Different partial point clouds for the same CAD Model:

  • High-quality complete point clouds:


[Citation]

If you find our code useful, please cite our paper:

@article{pan2021variational,
  title={Variational Relational Point Completion Network},
  author={Pan, Liang and Chen, Xinyi and Cai, Zhongang and Zhang, Junzhe and Zhao, Haiyu and Yi, Shuai and Liu, Ziwei},
  journal={arXiv preprint arXiv:2104.10154},
  year={2021}
}

[License]

Our code is released under Apache-2.0 License.


[Acknowledgement]

We include the following PyTorch 3rd-party libraries:
[1] CD
[2] EMD
[3] MMDetection3D

We include the following algorithms:
[1] PCN
[2] ECG
[3] VRCNet
[4] DCP
[5] DeepGMR
[6] IDAM

Comments
  • Some problems about private test

    Some problems about private test

    1. Is the dataset for the private test the same as the dataset for the public test?
    2. How do we get feedback on the private test?
    3. What is the experimental environment of the private test (both software and hardware environment), do we need to configure it?
    opened by Dizzy-cell 7
  • ImportError cannot import name 'ball_query_ext' from 'mm3d_pn2.ops.ball_query'  急急急,在线等

    ImportError cannot import name 'ball_query_ext' from 'mm3d_pn2.ops.ball_query' 急急急,在线等

    当我运行 python train.py -c ./cfgs/vcrnet.yaml

    报错信息是这样的 :ImportError cannot import name 'ball_query_ext' from 'mm3d_pn2.ops.ball_query'

    下面是 'mm3d_pn2.ops.ball_query 这个文件夹中的内容,里面确实没有 'ball_query_ext' 这个文件

    utils metrics mm3d_pn2 ops ball_query src ball_query.cpp ball_query_cuda.cu init.py ball_query.py

    ball_query.py 这个文件里面有这么一条导入语句:

    import torch
    from torch.autograd import Function
    
    from . import ball_query_ext  // 在当前目录中没有这个模块文件,请问这个错误该咋解决呢?
    
    
    class BallQuery(Function):
        """Ball Query.
    
        Find nearby points in spherical space.
        """
    
        @staticmethod
        def forward(ctx, min_radius: float, max_radius: float, sample_num: int,
                    xyz: torch.Tensor, center_xyz: torch.Tensor) -> torch.Tensor:
                ................
                ...................
                .................
    

    from . import ball_query_ext // 在当前目录中没有这个 ball_query_ext.py 模块文件,请问这个错误该咋解决呢?

    opened by xjtuzhjm 7
  • About the GPU usage and training time of several methods in registration.

    About the GPU usage and training time of several methods in registration.

    Hello, I tried to test the code for point cloud registration in Benchmark, but I found that the GPU utilization for these methods is very low (jumping between 0% and 20%, and the training process is also very slow), I used multiple GPUs, do you use a single one for the training process or? Can you provide us with your hardware environment and the approximate time required for a full training or an epoch? I'm not sure if it's my environment or code problem, I'd like to get your help.

    Two test environments: a). cuda10.2, GPU 2080Ti, pytorch1.5.0; b). cuda11.1, GPU 3090, pytorch1.7.0.

    opened by MaxChanger 7
  • I may be stupid but where can I find the best_cd_t_network.pth

    I may be stupid but where can I find the best_cd_t_network.pth

    After reading your paper about VRCNet, I found this net is based on a prior pth. But I cannot find best_cd_t_network.pth in this repo. I found pretrained_vrcnet_2048.pth in VRCNet repo, but it seemed not working... help me...

    opened by xkevas24 4
  • Questions about pre-trained weight and so on

    Questions about pre-trained weight and so on

    Hello, I have three questions about this competition.

    1. Will it be illegal to use the pre-trained weight from other methods, like "PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers"? The result could be better if I train the network based on the pre-trained PCN/C3D weight.
    2. Is the submission on the competition website still effective during the private submission stage?
    3. Can we only give the private submission without showing it on the leaderboard?

    Thank you very much!

    opened by ganddam 3
  • There may be some bugs in the ranking order of the point cloud registration leaderboard?

    There may be some bugs in the ranking order of the point cloud registration leaderboard?

    I don't know if it is correct, I think there may be some bugs in the ranking order of the point cloud registration leaderboard? It looks like it’s currently sorted by submission time, not by some kind of metric.

    opened by MaxChanger 2
  • Missing super resolution dataset

    Missing super resolution dataset

    Hi,

    thanks for releasing the code and dataset. However, on the paper you provide results at different resolutions (2.048 - 4.096 - 8.192 16.384), which are not present in the current released dataset. Why have they been removed ?

    Thanks Riccardo

    opened by rspezialetti 2
  • About the trained model for baseline methods

    About the trained model for baseline methods

    Hi, @paul007pl ,

    Thanks for releasing the code for the benchmark and baseline models. Do you also provide the trained models (checkpoints) for baseline methods (e.g. [1] PCN; [2] ECG; [3] VRCNet)? It'll be useful for having consistent comparison results.

    Thanks~

    opened by amiltonwong 2
  • Is it possible to provide your script for rendering partial point cloud (or some hints)?

    Is it possible to provide your script for rendering partial point cloud (or some hints)?

    Hi, thank you for your great work! I'm also recently trying to render some partial point clouds from mesh in my own dataset. I'm using pyrender but it turns out to be super slow. I just wonder if it's possible for you to provide the rendering script? Or may I ask how did you render those partial point clouds (e.g. what software/library did you use) and how long does it take? Thanks!

    opened by Wuziyi616 1
  • Questions about  private test.

    Questions about private test.

    “We rank each method mainly based on the performance, and you will receive the final ranking list. ” @paul007pl, When would we receive the final ranking list?

    opened by Dizzy-cell 0
  • Question about reply email for Private test

    Question about reply email for Private test

    I have sent the email of Pravite test to [email protected] and [email protected] yesterday, but I didn't recive a reply. 1. Is it normal to not receive the email about reply? 2. Is there any place to confirm that our documents were successfully received?

    opened by wangql16 0
  • Training on multiple gpus

    Training on multiple gpus

    Dear Authors,

    I noticed that you train the model for completion "using the Adam optimizer with initial learning rate 1e−4 (decayed by 0.7 every 40 epochs) and batch size 32 by NVIDIA TITAN Xp GPU" mentioned in your paper. Did you mean that you used a single GPU for training or more GPUs?

    Thanks for your work and look forward to your favourable reply.

    opened by chenzhik 0
  • ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

    ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

    While trying to run-> pip install git+https://github.com/open-mmlab/mmdetection.git in the mm3d_pn2 directory, I am getting the following error, how can i fix this?

    $ pip install git+https://github.com/open-mmlab/mmdetection.git Collecting git+https://github.com/open-mmlab/mmdetection.git Cloning https://github.com/open-mmlab/mmdetection.git to c:\users\sshre34\appdata\local\temp\pip-req-build-gmxamwua Running command git clone --filter=blob:none --quiet https://github.com/open-mmlab/mmdetection.git 'C:\Users\sshre34\AppData\Local\Temp\pip-req-build-gmxamwua' Resolved https://github.com/open-mmlab/mmdetection.git to commit 94afcdc6aa1e2c849c9d8334ec7730bb52b17922 Preparing metadata (setup.py) ... done Requirement already satisfied: matplotlib in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from mmdet==2.27.0) (3.5.3) Requirement already satisfied: numpy in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from mmdet==2.27.0) (1.21.5) Collecting pycocotools Using cached pycocotools-2.0.6.tar.gz (24 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: scipy in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from mmdet==2.27.0) (1.7.3) Requirement already satisfied: six in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from mmdet==2.27.0) (1.16.0) Collecting terminaltables Using cached terminaltables-3.1.10-py2.py3-none-any.whl (15 kB) Requirement already satisfied: python-dateutil>=2.7 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (2.8.2) Requirement already satisfied: pillow>=6.2.0 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (9.3.0) Requirement already satisfied: fonttools>=4.22.0 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (4.38.0) Requirement already satisfied: packaging>=20.0 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (22.0) Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (1.4.4) Requirement already satisfied: cycler>=0.10 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (0.11.0) Requirement already satisfied: pyparsing>=2.2.1 in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from matplotlib->mmdet==2.27.0) (3.0.9) Requirement already satisfied: typing-extensions in c:\users\sshre34\anaconda3\envs\mvp\lib\site-packages (from kiwisolver>=1.0.1->matplotlib->mmdet==2.27.0) (4.4.0) Building wheels for collected packages: mmdet, pycocotools Building wheel for mmdet (setup.py) ... done Created wheel for mmdet: filename=mmdet-2.27.0-py3-none-any.whl size=1468682 sha256=ee7389458986d8bcfb5c5f486cee48393c239c2200caf362b0b4d31e5a6348ff Stored in directory: C:\Users\sshre34\AppData\Local\Temp\pip-ephem-wheel-cache-iu2x_jfa\wheels\8d\1d\4c\5ba147e9294578f513772158a509161804e08d1b5f62e95705 Building wheel for pycocotools (pyproject.toml) ... error error: subprocess-exited-with-error

    × Building wheel for pycocotools (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [16 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-cpython-37 creating build\lib.win-amd64-cpython-37\pycocotools copying pycocotools\coco.py -> build\lib.win-amd64-cpython-37\pycocotools copying pycocotools\cocoeval.py -> build\lib.win-amd64-cpython-37\pycocotools copying pycocotools\mask.py -> build\lib.win-amd64-cpython-37\pycocotools copying pycocotools_init_.py -> build\lib.win-amd64-cpython-37\pycocotools running build_ext cythoning pycocotools/_mask.pyx to pycocotools_mask.c building 'pycocotools._mask' extension C:\Users\sshre34\AppData\Local\Temp\pip-build-env-02b_xbrm\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\sshre34\AppData\Local\Temp\pip-install-3k0didqo\pycocotools_c99855141cd14b4c944e7ec99dcb18aa\pycocotools_mask.pyx tree = Parsing.p_module(s, pxd, full_module_name) error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pycocotools Successfully built mmdet Failed to build pycocotools ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

    opened by shreeja1 0
  • about train.log

    about train.log

    thank you for your amazing work! i meet a problem and i can not solve it, could you give me some advice? Traceback (most recent call last): File "/raid/MVP_Benchmark/completion/train.py", line 211, in logging.basicConfig(level=logging.INFO, handlers=[logging.FileHandler(os.path.join(log_dir, 'train.log')), File "/home/anaconda3/envs/mvp/lib/python3.7/logging/init.py", line 1087, in init StreamHandler.init(self, self._open()) File "/home/anaconda3/envs/mvp/lib/python3.7/logging/init.py", line 1116, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) FileNotFoundError: [Errno 2] No such file or directory: '/raid/jiangzw/MVP_Benchmark/completion/log/vrcnet_cd_debug_2021-06-18T16:36:20/train.log'

    opened by JiangZhenW 0
  • Tensor full of nan during the train of vrcnet

    Tensor full of nan during the train of vrcnet

    Hy, thanks for your work! During the training of the vrcnet, the following problem occurs:

    nan vrcnet

    Especially, the problem occurs whithin the function edge_preserving_sampling, which is called inside the forward of the SA_SKN_Res_encoder:

    ds_features, p_idx, pn_idx, ds_points = edge_preserve_sampling(features, points, sample_num, k)

    and the feature tensors become full nan tensors. Is there a way to solve this problem? Did I do something wrong during the training?

    opened by acavalli1996 0
Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021) An efficient PyTorch library for Point Cloud Completion.

Microsoft 119 Jan 2, 2023
[ICCV 2021 Oral] SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer

This repository contains the source code for the paper SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer (ICCV 2021 Oral). The project page is here.

AllenXiang 65 Dec 26, 2022
(CVPR 2022 - oral) Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry

Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry Official implementation of the paper Multi-View Depth Est

Bae, Gwangbin 138 Dec 28, 2022
Registration Loss Learning for Deep Probabilistic Point Set Registration

RLLReg This repository contains a Pytorch implementation of the point set registration method RLLReg. Details about the method can be found in the 3DV

Felix Järemo Lawin 35 Nov 2, 2022
Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture

MS-SVConv : 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning Compute features for 3D point cloud registration

null 42 Jul 25, 2022
Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

null 75 Nov 24, 2022
GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration

Geometric Transformer for Fast and Robust Point Cloud Registration PyTorch imple

Zheng Qin 220 Jan 5, 2023
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

null 153 Dec 14, 2022
You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors

You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors In this paper, we propose a novel local descriptor-based fra

Haiping Wang 80 Dec 15, 2022
Point Cloud Registration using Representative Overlapping Points.

Point Cloud Registration using Representative Overlapping Points (ROPNet) Abstract 3D point cloud registration is a fundamental task in robotics and c

ZhuLifa 36 Dec 16, 2022
Cascading Feature Extraction for Fast Point Cloud Registration (BMVC 2021)

Cascading Feature Extraction for Fast Point Cloud Registration This repository contains the source code for the paper [Arxive link comming soon]. Meth

null 7 May 26, 2022
PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

null 76 Jan 3, 2023
Point Cloud Registration Network

PCRNet: Point Cloud Registration Network using PointNet Encoding Source Code Author: Vinit Sarode and Xueqian Li Paper | Website | Video | Pytorch Imp

ViNiT SaRoDe 59 Nov 19, 2022
[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

Xumin Yu 317 Dec 26, 2022
GEP (GDB Enhanced Prompt) - a GDB plug-in for GDB command prompt with fzf history search, fish-like autosuggestions, auto-completion with floating window, partial string matching in history, and more!

GEP (GDB Enhanced Prompt) GEP (GDB Enhanced Prompt) is a GDB plug-in which make your GDB command prompt more convenient and flexibility. Why I need th

Alan Li 23 Dec 21, 2022
the code of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021)

RMA-Net This repo is the implementation of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021). Paper

Wanquan Feng 205 Nov 9, 2022
3D AffordanceNet is a 3D point cloud benchmark consisting of 23k shapes from 23 semantic object categories, annotated with 56k affordance annotations and covering 18 visual affordance categories.

3D AffordanceNet This repository is the official experiment implementation of 3D AffordanceNet benchmark. 3D AffordanceNet is a 3D point cloud benchma

null 49 Dec 1, 2022
Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Blender add-on: Camera additions In 3D view, it adds these actions to the View|Cameras menu: View → Camera : set the current camera to the 3D view Vie

German Bauer 11 Feb 8, 2022
Implementation of the "PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences" paper.

PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences Introduction Point cloud sequences are irregular and unordered in the spatial dimen

Hehe Fan 63 Dec 9, 2022