Pytorch implementation of PCT: Point Cloud Transformer

Overview

PCT: Point Cloud Transformer

This is a Pytorch implementation of PCT: Point Cloud Transformer.

Paper link: https://arxiv.org/pdf/2012.09688.pdf

Requirements

python >= 3.7

pytorch >= 1.6

h5py

scikit-learn

and

pip install pointnet2_ops_lib/.

The code is from https://github.com/erikwijmans/Pointnet2_PyTorch https://github.com/WangYueFt/dgcnn and https://github.com/MenghaoGuo/PCT

Models

We get an accuracy of 93.2% on the ModelNet40(http://modelnet.cs.princeton.edu/) validation dataset

The path of the model is in ./checkpoints/train/models/model.t7

Example training and testing

# train
python main.py --exp_name=train --num_points=1024 --use_sgd=True --batch_size 32 --epochs 250 --lr 0.0001

# test
python main.py --exp_name=test --num_points=1024 --use_sgd=True --eval=True --model_path=checkpoints/best/models/model.t7 --test_batch_size 8

Citation

If it is helpful for your work, please cite this paper:

@misc{guo2020pct,
      title={PCT: Point Cloud Transformer}, 
      author={Meng-Hao Guo and Jun-Xiong Cai and Zheng-Ning Liu and Tai-Jiang Mu and Ralph R. Martin and Shi-Min Hu},
      year={2020},
      eprint={2012.09688},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Comments
  • ninja问题

    ninja问题

    请问一下这种情况该怎么解决,运行的时候报了这个错误,已经安装了ninja还是不可以

    roup_points_gpu.cuda.o FAILED: group_points_gpu.cuda.o /usr/bin/nvcc --generate-dependencies-with-compile --dependency-output group_points_gpu.cuda.o.d -DTORCH_EXTENSION_NAME=ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -I/home/lin/program/PCT_Pytorch-main/pointnet2_ops_lib/pointnet2_ops/ext-src/include -isystem /home/lin/anaconda3/lib/python3.8/site-packages/torch/include -isystem /home/lin/anaconda3/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/lin/anaconda3/lib/python3.8/site-packages/torch/include/TH -isystem /home/lin/anaconda3/lib/python3.8/site-packages/torch/include/THC -isystem /home/lin/anaconda3/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_37,code=compute_37 -gencode=arch=compute_37,code=sm_37 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_62,code=sm_62 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -O3 -Xfatbin -compress-all -std=c++14 -c /home/lin/program/PCT_Pytorch-main/pointnet2_ops_lib/pointnet2_ops/_ext-src/src/group_points_gpu.cu -o group_points_gpu.cuda.o nvcc fatal : Unknown option '-generate-dependencies-with-compile' ninja: build stopped: subcommand failed.

    opened by trra1988 7
  • ImportError: No module named '_ext'

    ImportError: No module named '_ext'

    ModuleNotFoundError: No module named '_ext'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "main.py", line 9, in from model import Pct File "/home/cx/NewDisk/xxl/PCT_Pytorch-main/model.py", line 4, in from util import sample_and_group File "/home/cx/NewDisk/xxl/PCT_Pytorch-main/util.py", line 3, in from pointnet2_ops_lib.pointnet2_ops import pointnet2_utils File "/home/cx/NewDisk/xxl/PCT_Pytorch-main/pointnet2_ops_lib/pointnet2_ops/init.py", line 1, in from . import pointnet2_modules File "/home/cx/NewDisk/xxl/PCT_Pytorch-main/pointnet2_ops_lib/pointnet2_ops/pointnet2_modules.py", line 6, in from . import pointnet2_utils File "/home/cx/NewDisk/xxl/PCT_Pytorch-main/pointnet2_ops_lib/pointnet2_ops/pointnet2_utils.py", line 30, in with_cuda=True, File "/home/cx/NewDisk/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 661, in load is_python_module) File "/home/cx/NewDisk/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 841, in _jit_compile return _import_module_from_library(name, build_directory, is_python_module) File "/home/cx/NewDisk/anaconda3/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1048, in _import_module_from_library file, path, description = imp.find_module(module_name, [path]) File "/home/cx/NewDisk/anaconda3/lib/python3.6/imp.py", line 297, in find_module raise ImportError(_ERR_MSG.format(name), name=name) ImportError: No module named '_ext'

    opened by zcxxlshirley 3
  • About the consumption of memory when training

    About the consumption of memory when training

    I got this when training modelnet40 dataset; RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 3.94 GiB total capacity; 2.98 GiB already allocated; 14.25 MiB free; 3.00 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

    And it dosent work, even if I reduce the batchsize(2). Can I have any other options, besides a new GPU with more memory? How much memory do you use when training? Thanks a lot!

    opened by KevinWisbay 1
  • The results on val datast of ModelNet40 is only about 45.91% !

    The results on val datast of ModelNet40 is only about 45.91% !

    Hi @Strawberry-Eat-Mango @uyzhang, Thanks for sharing the PyTorch implementation of PCT!

    I follow the instructions in this repo and train PCT from scratch on ModelNet40. First, I prepare the dataset. Then configure the training parameters, shown in Fig.1, and keep others default (6 2080ti devices are used, but it's weird that the last two occupy fewer memories, shown in Fig.2). Third, 'scheduler.step()' was added as other issues suggested to eliminate a warning.

    Fig. 1 1640876090(1)

    Fig. 2 1640876489(1)

    Everything looks good: the loss reduces and the accuracy increases during training. One exceptional point is that the accuracy increases slowly, e.g. Accuracy achieves only 29% after 200 epochs of training, and finally the accuracy stops at 45.91% in 250 epochs of training, shown in Fig.3 . Hence, the results are far from expectations.

    Fig. 3 1640876718(1)

    What are possible reasons? Can anyone give some tips or suggestions?

    Appreciate!

    opened by auniquesun 1
  • About the license

    About the license

    Dear authors,

    Thank you for sharing your code and contributions. I would like to use your code in my project, while there is no LICENSE present in your repository. May I ask in what ways we could use or modify the code, and it wouild be great if you could add the license to the repository.

    Thank you so much!

    opened by haotian-liu 1
  • unmet accuracy

    unmet accuracy

    Thanks for this implementation!

    I ran the test code with

    python main.py --exp_name=test --num_points=1024 --use_sgd=True --eval=True --model_path=checkpoints/best/models/model.t7 --test_batch_size=4
    

    (changed test_batch_size to 4 from 8 due to GPU memory constraints)

    But the accuracy is really bad:

    Test :: test acc: 0.483387, test avg acc: 0.455517
    

    The only thing i did slightly differently to the original code is that, I downloaded the h5 data manually instead of using the code, as otherwise i met certificate issues.

    Can anyone else replicate the 93% accuracy as reported in the repo?

    update setting --test_batch_size=1 achieves test acc: 0.932739, test avg acc: 0.899901 another observation, if test_batch_size > 1, prediction result is not reproducible / deterministic.

    I'm using pytorch 1.6.0

    opened by yifita 1
  • 请问,新模型训练后的测试出现如下问题?

    请问,新模型训练后的测试出现如下问题?

    RuntimeError: Error(s) in loading state_dict for DataParallel: Missing key(s) in state_dict: "module.pt_last.pos_xyz.weight", "module.pt_last.pos_xyz.bias". Unexpected key(s) in state_dict: "module.pt_last.conv2.weight", "module.pt_last.bn2.weight", "module.pt_last.bn2.bias", "module.pt_last.bn2.running_mean", "module.pt_last.bn2.running_var", "module.pt_last.bn2.num_batches_tracked".

    opened by swzaaaaaaa 0
  • About the _ext problems

    About the _ext problems

    Run to pointnet2_ When using utils.py, prompt userw arning:“Uable to load pointnet2_ OPD CPP extension. JIT compling”, and then the program is stuck here, and the run cursor keeps flashing. What's wrong? Thank you

    opened by AI-Hello 0
  • issue about install pointnet2_ops_lib

    issue about install pointnet2_ops_lib

    I'm new to this filed, when I tried to install pointnet2_ops_lib, I got this:

    (transformer) user002@user002-OptiPlex-7010:~/下载/PCT_Pytorch-main$ pip install pointnet2_ops_lib/. Processing ./pointnet2_ops_lib Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [12 lines of output] Traceback (most recent call last): File "", line 36, in File "", line 34, in File "/home/user002/下载/PCT_Pytorch-main/pointnet2_ops_lib/setup.py", line 34, in include_dirs=[osp.join(this_dir, _ext_src_root, "include")], File "/home/user002/miniconda3/envs/transformer/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 779, in CUDAExtension library_dirs += library_paths(cuda=True) File "/home/user002/miniconda3/envs/transformer/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 869, in library_paths if (not os.path.exists(_join_cuda_home(lib_dir)) and File "/home/user002/miniconda3/envs/transformer/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1783, in _join_cuda_home raise EnvironmentError('CUDA_HOME environment variable is not set. ' OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

    × Encountered error while generating package metadata. ╰─> See above for output.

    note: This is an issue with the package mentioned above, not pip. hint: See above for details.

    How can I solve it?

    opened by Chengjlzzz 0
  • Difference between the origin implemention

    Difference between the origin implemention

    Hi, thanks for your work! I found that there is some difference between your code and origin jittor implemention. In the class Point_Transformer_Last module. you don't input the xyz information, where the original implemention takes a position embedding. Have you compared the two results? Appreciate!

    opened by songtianhui 0
  • 你好,我在进行训练时遇到了这个问题,我使用的显卡型号是2080ti,torch版本为1.6.0

    你好,我在进行训练时遇到了这个问题,我使用的显卡型号是2080ti,torch版本为1.6.0

    Use SGD CUDA kernel failed : invalid device function void furthest_point_sampling_kernel_wrapper(int, int, int, const float*, float*, int*) at L:228 in pointnet2_ops/_ext-src/src/sampling_gpu.cu 段错误 (核心已转储)

    opened by liu-zikang 0
  • a warning

    a warning

    UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
    

    Is it ok?

    opened by zhenhao-huang 2
Owner
Yi_Zhang
Yi_Zhang
Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

null 75 Nov 24, 2022
Implementation of the "Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos" paper.

Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos Introduction Point cloud videos exhibit irregularities and lack of or

Hehe Fan 101 Dec 29, 2022
[ICCV 2021 Oral] SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer

This repository contains the source code for the paper SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer (ICCV 2021 Oral). The project page is here.

AllenXiang 65 Dec 26, 2022
Implementation of the "PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences" paper.

PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences Introduction Point cloud sequences are irregular and unordered in the spatial dimen

Hehe Fan 63 Dec 9, 2022
Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021) An efficient PyTorch library for Point Cloud Completion.

Microsoft 119 Jan 2, 2023
Synthetic LiDAR sequential point cloud dataset with point-wise annotations

SynLiDAR dataset: Learning From Synthetic LiDAR Sequential Point Cloud This is official repository of the SynLiDAR dataset. For technical details, ple

null 78 Dec 27, 2022
GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration

Geometric Transformer for Fast and Robust Point Cloud Registration PyTorch imple

Zheng Qin 220 Jan 5, 2023
Stratified Transformer for 3D Point Cloud Segmentation (CVPR 2022)

Stratified Transformer for 3D Point Cloud Segmentation Xin Lai*, Jianhui Liu*, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, Jiaya Jia

DV Lab 195 Jan 1, 2023
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

null 153 Dec 14, 2022
PyTorch implementation for View-Guided Point Cloud Completion

PyTorch implementation for View-Guided Point Cloud Completion

null 22 Jan 4, 2023
PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

null 76 Jan 3, 2023
Pytorch implementation of Straight Sampling Network For Point Cloud Learning (ICIP2021).

Pytorch code for SS-Net This is a pytorch implementation of Straight Sampling Network For Point Cloud Learning (ICIP2021). Environment Code is tested

Sun Ran 1 May 18, 2022
Official pytorch implementation of "DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion"

DSPoint Official implementation of "DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion". Paper link: https://arxiv.org/abs/2111.10

Ziyao Zeng 10 Nov 24, 2021
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 7, 2022
Official implementation of YOGO for Point-Cloud Processing

You Only Group Once: Efficient Point-Cloud Processing with Token Representation and Relation Inference Module By Chenfeng Xu, Bohan Zhai, Bichen Wu, T

Chenfeng Xu 67 Dec 20, 2022
Implementation of the Point Transformer layer, in Pytorch

Point Transformer - Pytorch Implementation of the Point Transformer self-attention layer, in Pytorch. The simple circuit above seemed to have allowed

Phil Wang 501 Jan 3, 2023
VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

Jiezhang Cao 225 Nov 13, 2022
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 5, 2022
Point-NeRF: Point-based Neural Radiance Fields

Point-NeRF: Point-based Neural Radiance Fields Project Sites | Paper | Primary c

Qiangeng Xu 662 Jan 1, 2023