Score-Based Point Cloud Denoising (ICCV'21)

Overview

Score-Based Point Cloud Denoising (ICCV'21)

teaser

[Paper] https://arxiv.org/abs/2107.10981

Installation

Recommended Environment

The code has been tested in the following environment:

Package Version Comment
PyTorch 1.9.0
point_cloud_utils 0.18.0 For evaluation only. It loads meshes to compute point-to-mesh distances.
pytorch3d 0.5.0 For evaluation only. It computes point-to-mesh distances.
pytorch-cluster 1.5.9 We only use fps (farthest point sampling) to merge denoised patches.

Install via Conda (PyTorch 1.9.0 + CUDA 11.1)

conda env create -f env.yml
conda activate score-denoise

Install Manually

conda create --name score-denoise python=3.8
conda activate score-denoise

conda install pytorch==1.9.0 torchvision==0.10.0 cudatoolkit=11.1 -c pytorch -c nvidia

conda install -c conda-forge tqdm scipy scikit-learn pyyaml easydict tensorboard pandas

# point_cloud_utils
conda install -c conda-forge point_cloud_utils==0.18.0

# Pytorch3d
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c pytorch3d pytorch3d==0.5.0

# pytorch-scatter
conda install -c pyg pytorch-cluster==1.5.9

Datasets

Download link: https://drive.google.com/drive/folders/1--MvLnP7dsBgBZiu46H0S32Y1eBa_j6P?usp=sharing

Please extract data.zip to data folder.

Denoise

Reproduce Paper Results

# PUNet dataset, 10K Points
python test.py --dataset PUNet --resolution 10000_poisson --noise 0.01 --niters 1
python test.py --dataset PUNet --resolution 10000_poisson --noise 0.02 --niters 1
python test.py --dataset PUNet --resolution 10000_poisson --noise 0.03 --niters 2
# PUNet dataset, 50K Points
python test.py --dataset PUNet --resolution 50000_poisson --noise 0.01 --niters 1
python test.py --dataset PUNet --resolution 50000_poisson --noise 0.02 --niters 1
python test.py --dataset PUNet --resolution 50000_poisson --noise 0.03 --niters 2

Denoise Regular-Size Point Clouds (≤ 50K Points)

python test_single.py --input_xyz <input_xyz_path> --output_xyz <output_xyz_path>

You may also barely run python test_single.py to see a quick example.

Denoise Large Point Clouds (> 50K Points)

python test_large.py --input_xyz <input_xyz_path> --output_xyz <output_xyz_path>

You may also barely run python test_large.py to see a quick example.

Train

python train.py

Please find tunable parameters in the script.

Citation

@InProceedings{Luo_2021_ICCV,
    author    = {Luo, Shitong and Hu, Wei},
    title     = {Score-Based Point Cloud Denoising},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {4583-4592}
}
Comments
  • train results

    train results

    D:\anaconda\anaconda\envs\score\python.exe D:/code/score-denoise-main/score-denoise-main/test.py [2022-07-16 20:04:07,207::test::INFO] [ARGS::ckpt] './pretrained/ckpt.pt' [2022-07-16 20:04:07,207::test::INFO] [ARGS::input_root] './data/examples' [2022-07-16 20:04:07,208::test::INFO] [ARGS::output_root] './data/results' [2022-07-16 20:04:07,208::test::INFO] [ARGS::dataset_root] './data' [2022-07-16 20:04:07,208::test::INFO] [ARGS::dataset] 'PCNet' [2022-07-16 20:04:07,208::test::INFO] [ARGS::tag] '' [2022-07-16 20:04:07,208::test::INFO] [ARGS::resolution] '10000_poisson' [2022-07-16 20:04:07,208::test::INFO] [ARGS::noise] '0.01' [2022-07-16 20:04:07,208::test::INFO] [ARGS::device] 'cuda' [2022-07-16 20:04:07,208::test::INFO] [ARGS::seed] 2020 [2022-07-16 20:04:07,208::test::INFO] [ARGS::ld_step_size] None [2022-07-16 20:04:07,208::test::INFO] [ARGS::ld_step_decay] 0.95 [2022-07-16 20:04:07,209::test::INFO] [ARGS::ld_num_steps] 30 [2022-07-16 20:04:07,209::test::INFO] [ARGS::seed_k] 3 [2022-07-16 20:04:07,209::test::INFO] [ARGS::niters] 1 [2022-07-16 20:04:07,209::test::INFO] [ARGS::denoise_knn] 4 [2022-07-16 20:04:10,298::test::INFO] ld_step_size = 0.20000000 [2022-07-16 20:04:10,345::test::INFO] boxunion [2022-07-16 20:04:18,952::test::INFO] box_push [2022-07-16 20:04:24,497::test::INFO] column_head [2022-07-16 20:04:30,082::test::INFO] cylinder [2022-07-16 20:04:35,609::test::INFO] dragon [2022-07-16 20:04:41,134::test::INFO] galera [2022-07-16 20:04:46,660::test::INFO] happy [2022-07-16 20:04:52,189::test::INFO] icosahedron [2022-07-16 20:04:57,717::test::INFO] netsuke [2022-07-16 20:05:03,248::test::INFO] star_smooth Loading: 100%|██████████| 10/10 [00:00<00:00, 33.74it/s] Loading: 100%|██████████| 10/10 [00:00<00:00, 34.59it/s] Loading: 100%|██████████| 10/10 [00:06<00:00, 1.48it/s] Evaluate: 100%|██████████| 10/10 [00:11<00:00, 1.18s/it] [2022-07-16 20:05:27,919::test::INFO] cd_sph p2f boxunion 0.000311 0.000019 box_push 0.000357 0.000229 column_head 0.000530 0.000344 cylinder 0.000321 0.000067 dragon 0.000317 0.000208 galera 0.000383 0.000300 happy 0.000302 0.000217 icosahedron 0.000331 0.000065 netsuke 0.000242 0.000148 star_smooth 0.000276 0.000110 [2022-07-16 20:05:27,919::test::INFO] Mean cd_sph 0.000336946515 p2f 0.000170847624

    Process finished with exit code 0

    I used the hyperparameter you saved, but the P2M accuracy is abnormal. P2M in the output are quite different from those given in your paper.May I ask why this is?

    opened by tang-y-q 8
  • Supervised loss function

    Supervised loss function

    Hi @luost26,

    Thank you for sharing your implementation! I just have a question about the supervised (and self-supervised) loss function. In https://github.com/luost26/score-denoise/blob/main/models/denoise.py line 77, what is the purpose of self.dsm_sigma? I was not able to find this in the paper.

    Furthermore, in equation 3 of the main paper, you take the expectation with respect to the distribution N(x_i). But in the code this is a straightforward average so is this a uniform distribution? score-based-denoising-question

    Thank you! D.

    opened by ddsediri 4
  • dataset

    dataset

    In the paper, 20 meshes of PU dataset were used to train, but in the code, there are 40 meshes. Can it influence the result? I have retrain the network and get worse results.

    opened by rsy6318 4
  • Results on a large number of points

    Results on a large number of points

    Hello:) I launched your model on a large number of points (200 thousand +-) on an airplane model, but there were no improvements to the model. On the left, the point cloud before the change, on the right after the change. I also tried to reduce the number of points to 70 thousand and there was no result either.

    1. Why does the model not give results on a large number of points?
    2. Does it make sense to try to train the model on your loyal data? image image
    opened by Siziff 3
  • About dataset production and baseline

    About dataset production and baseline

    Thank you for your excellent work!

    1. I want to create my own dataset. I wonder if I just need to provide a clean point cloud? I found that this model seems to include a function that adds noise to the point cloud.

    2. For some baselines in the article, such as MRPCA and GLR, I found that there seems to be no publicly available code for these methods. Did you implement them yourself?

    opened by suetme 1
  • unsupervised adaptation of model

    unsupervised adaptation of model

    Hi, great work!

    I am very curious about the unsupervised adaptation. Unfortunately, the supplementary material you mentioned is not available in the paper(v3) in arxiv.

    It would very nice if you could share the supplementary part. Looking forward to the implementation as well.

    opened by huidiwiwi 1
  • RuntimeError: Given groups=1, weight of size [128, 63, 1], expected input[100000, 66, 4] to have 63 channels, but got 66 channels instead

    RuntimeError: Given groups=1, weight of size [128, 63, 1], expected input[100000, 66, 4] to have 63 channels, but got 66 channels instead

     python test_large.py 
    
    Running KMeans to construct clusters...
    Denoise Clusters:   0%|                                                                                                                                                                                                                                    | 0/69 [00:00<?, ?it/s]
    Traceback (most recent call last):
    
    
    RuntimeError: Given groups=1, weight of size [128, 63, 1], expected input[100000, 66, 4] to have 63 channels, but got 66 channels instead
    
    
    opened by eyildiz-ugoe 1
  • Is it possible to get the score separately?

    Is it possible to get the score separately?

    Hi, I am working on evaluating the point cloud, and I think maybe it is possible to use the “score“ idea to give each point a “convincing score”. So I have read your code, but actually it is a little hard to understand the whole part. I want to ask whether it is possible to make score separately be output? If I understand it correctly, the first step is to use the feature net and use the feature net output with the original point could to get the score. When is the score calculated out? The final score will be like the equation 4 in the thesis, and is it actually associated to each point? Thanks in advance.

    opened by Gycy2041 0
Owner
Shitong Luo
Undergraduate @ PKU
Shitong Luo
Source code for the paper "TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations"

TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations Created by Jiahao Pang, Duanshun Li, and Dong Tian from InterDigital In

InterDigital 21 Dec 29, 2022
Estimation of the CEFR complexity score of a given word, sentence or text.

NLP-Swedish … allows to estimate CEFR (Common European Framework of References) complexity score of a given word, sentence or text. CEFR scores come f

null 3 Apr 30, 2022
An easier way to build neural search on the cloud

An easier way to build neural search on the cloud Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g

Jina AI 17.1k Jan 9, 2023
An easier way to build neural search on the cloud

An easier way to build neural search on the cloud Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g

Jina AI 2.3k Feb 18, 2021
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 B) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 Billion Parameters) on a single 16 GB VRAM V100 Google Cloud instance with Huggingfa

null 289 Jan 6, 2023
Implemented shortest-circuit disambiguation, maximum probability disambiguation, HMM-based lexical annotation and BiLSTM+CRF-based named entity recognition

Implemented shortest-circuit disambiguation, maximum probability disambiguation, HMM-based lexical annotation and BiLSTM+CRF-based named entity recognition

null 0 Feb 13, 2022
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

Rasa Open Source Rasa is an open source machine learning framework to automate text-and voice-based conversations. With Rasa, you can build contextual

Rasa 15.3k Dec 30, 2022
A natural language modeling framework based on PyTorch

Overview PyText is a deep-learning based NLP modeling framework built on PyTorch. PyText addresses the often-conflicting requirements of enabling rapi

Facebook Research 6.4k Dec 27, 2022
Summarization module based on KoBART

KoBART-summarization Install KoBART pip install git+https://github.com/SKT-AI/KoBART#egg=kobart Requirements pytorch==1.7.0 transformers==4.0.0 pytor

seujung hwan, Jung 148 Dec 28, 2022
CCKS-Title-based-large-scale-commodity-entity-retrieval-top1

- 基于标题的大规模商品实体检索top1 一、任务介绍 CCKS 2020:基于标题的大规模商品实体检索,任务为对于给定的一个商品标题,参赛系统需要匹配到该标题在给定商品库中的对应商品实体。 输入:输入文件包括若干行商品标题。 输出:输出文本每一行包括此标题对应的商品实体,即给定知识库中商品 ID,

null 43 Nov 11, 2022
PhoNLP: A BERT-based multi-task learning toolkit for part-of-speech tagging, named entity recognition and dependency parsing

PhoNLP is a multi-task learning model for joint part-of-speech (POS) tagging, named entity recognition (NER) and dependency parsing. Experiments on Vietnamese benchmark datasets show that PhoNLP produces state-of-the-art results, outperforming a single-task learning approach that fine-tunes the pre-trained Vietnamese language model PhoBERT for each task independently.

VinAI Research 109 Dec 2, 2022
An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.

GPT-NeoX An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hun

EleutherAI 3.1k Jan 8, 2023
Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

Trankit: A Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing Trankit is a light-weight Transformer-based Pyth

null 652 Jan 6, 2023
Transformer-based Text Auto-encoder (T-TA) using TensorFlow 2.

T-TA (Transformer-based Text Auto-encoder) This repository contains codes for Transformer-based Text Auto-encoder (T-TA, paper: Fast and Accurate Deep

Jeong Ukjae 13 Dec 13, 2022
NLP Core Library and Model Zoo based on PaddlePaddle 2.0

PaddleNLP 2.0拥有丰富的模型库、简洁易用的API与高性能的分布式训练的能力,旨在为飞桨开发者提升文本建模效率,并提供基于PaddlePaddle 2.0的NLP领域最佳实践。

null 6.9k Jan 1, 2023
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

Rasa Open Source Rasa is an open source machine learning framework to automate text-and voice-based conversations. With Rasa, you can build contextual

Rasa 15.3k Jan 3, 2023
Unsupervised text tokenizer for Neural Network-based text generation.

SentencePiece SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabu

Google 6.4k Jan 1, 2023