Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (CVPR 2022, Oral)

Related tags

Deep Learning point-cloud pytorch object-detection autonomous-driving 3d-object-detection ia-ssd
Overview

arXiv GitHub Stars visitors

Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (CVPR 2022, Oral)

This is the official implementation of IA-SSD (CVPR 2022), a simple and highly efficient point-based detector for 3D LiDAR point clouds. For more details, please refer to:

Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds
Yifan Zhang, Qingyong Hu*, Guoquan Xu, Yanxin Ma, Jianwei Wan, Yulan Guo

[Paper] [Video]

Getting Started

Installation

a. Clone this repository

git clone https://github.com/yifanzhang713/IA-SSD.git && cd IA-SSD

b. Configure the environment

We have tested this project with the following environments:

  • Ubuntu18.04/20.04
  • Python = 3.7
  • PyTorch = 1.1
  • CUDA = 10.0
  • CMake >= 3.13
  • spconv = 1.0 
    # install spconv=1.0 library
git clone https://github.com/yifanzhang713/spconv1.0.git
cd spconv1.0
sudo apt-get install libboostall-dev
python setup.py bdist_wheel
pip install ./dist/spconv-1.0*   # wheel file name may be different
cd ..

*You are encouraged to try to install higher versions above, please refer to the official github repository for more information. Note that the maximum number of parallel frames during inference might be slightly decrease due to the larger initial GPU memory footprint with updated Pytorch version.

c. Install pcdet toolbox.

pip install -r requirements.txt
python setup.py develop

d. Prepare the datasets.

Download the official KITTI with road planes and Waymo datasets, then organize the unzipped files as follows:

IA-SSD
├── data
│   ├── kitti
│   │   ├── ImageSets
│   │   ├── training
│   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes)
│   │   ├── testing
│   │   ├── calib & velodyne & image_2
│   ├── waymo
│   │   │── ImageSets
│   │   │── raw_data
│   │   │   │── segment-xxxxxxxx.tfrecord
|   |   |   |── ...
|   |   |── waymo_processed_data_v0_5_0
│   │   │   │── segment-xxxxxxxx/
|   |   |   |── ...
│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1/
│   │   │── waymo_processed_data_v0_5_0_waymo_dbinfos_train_sampled_1.pkl
│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1_global.npy (optional)
│   │   │── waymo_processed_data_v0_5_0_infos_train.pkl (optional)
│   │   │── waymo_processed_data_v0_5_0_infos_val.pkl (optional)
├── pcdet
├── tools

Generate the data infos by running the following commands:

# KITTI dataset
python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

# Waymo dataset
python -m pcdet.datasets.waymo.waymo_dataset --func create_waymo_infos \
    --cfg_file tools/cfgs/dataset_configs/waymo_dataset.yaml

Quick Inference

We provide the pre-trained weight file so you can just run with that:

cd tools 
# To achieve fully GPU memory footprint (NVIDIA RTX2080Ti, 11GB).
python test.py --cfg_file cfgs/kitti_models/IA-SSD.yaml --batch_size 100 \
    --ckpt IA-SSD.pth --set MODEL.POST_PROCESSING.RECALL_MODE 'speed'

# To reduce the pressure on the CPU during preprocessing, a suitable batchsize is recommended, e.g. 16. (Over 5 batches per second on RTX2080Ti)
python test.py --cfg_file cfgs/kitti_models/IA-SSD.yaml --batch_size 16 \
    --ckpt IA-SSD.pth --set MODEL.POST_PROCESSING.RECALL_MODE 'speed'
  • Then detailed inference results can be found here.

Training

The configuration files are in tools/cfgs/kitti_models/IA-SSD.yaml and tools/cfgs/waymo_models/IA-SSD.yaml, and the training scripts are in tools/scripts.

Train with single or multiple GPUs: (e.g., KITTI dataset)

python train.py --cfg_file cfgs/kitti_models/IA-SSD.yaml

# or 

sh scripts/dist_train.sh ${NUM_GPUS} --cfg_file cfgs/kitti_models/IA-SSD.yaml

Evaluation

Evaluate with single or multiple GPUs: (e.g., KITTI dataset)

python test.py --cfg_file cfgs/kitti_models/IA-SSD.yaml  --batch_size ${BATCH_SIZE} --ckpt ${PTH_FILE}

# or

sh scripts/dist_test.sh ${NUM_GPUS} \
    --cfg_file cfgs/kitti_models/IA-SSD.yaml --batch_size ${BATCH_SIZE} --ckpt ${PTH_FILE}

Experimental results

KITTI dataset

Quantitative results of different approaches on KITTI dataset (test set):

Qualitative results of our IA-SSD on KITTI dataset:

z z
z z

Quantitative results of different approaches on Waymo dataset (validation set):

Qualitative results of our IA-SSD on Waymo dataset:

z z
z z

Quantitative results of different approaches on ONCE dataset (validation set):

Qualitative result of our IA-SSD on ONCE dataset:

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{zhang2022not,
  title={Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds},
  author={Zhang, Yifan and Hu, Qingyong and Xu, Guoquan and Ma, Yanxin and Wan, Jianwei and Guo, Yulan},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2022}
}

Acknowledgement

  • This work is built upon the OpenPCDet (version 0.5), an open source toolbox for LiDAR-based 3D scene perception. Please refer to the official github repository for more information.

  • Parts of our Code refer to 3DSSD-pytorch-openPCDet library and the the recent work SASA.

License

This project is released under the Apache 2.0 license.

Related Repos

  1. RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds GitHub stars
  2. SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds GitHub stars
  3. 3D-BoNet: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds GitHub stars
  4. SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration GitHub stars
  5. SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds GitHub stars
  6. SoTA-Point-Cloud: Deep Learning for 3D Point Clouds: A Survey GitHub stars
Comments
  • Inconsistencies in inferencing results

    Inconsistencies in inferencing results

    The same frame point cloud is inputed into the model, but the result is different each time. For example. for the same frame point cloud, two cars are detected by the model for the first time, but three cars are detected by the model for the second time.The results are very erratic. Can you explain that, please?

    opened by JaydencoolCC 17
  • Question about the batch size setting when testing the model

    Question about the batch size setting when testing the model

    It seems that the batch size will affect the permformance when testing the model using the same pretrained weight file. Here's my result:

    • testing with batch_size=8
    Car [email protected], 0.70, 0.70:
bbox AP:96.2771, 90.1569, 89.5080
bev  AP:90.4018, 88.8616, 86.9882
3d   AP:89.3976, 79.5647, 78.4486
aos  AP:96.25, 90.10, 89.35
Car [email protected], 0.70, 0.70:
bbox AP:97.8890, 95.3317, 92.7613
bev  AP:94.7081, 91.3551, 88.8493
3d   AP:91.7842, 83.3762, 80.3495
aos  AP:97.86, 95.24, 92.57
Car [email protected], 0.50, 0.50:
bbox AP:96.2771, 90.1569, 89.5080
bev  AP:96.3634, 90.2276, 89.7638
3d   AP:96.3225, 90.2001, 89.6938
aos  AP:96.25, 90.10, 89.35
Car [email protected], 0.50, 0.50:
bbox AP:97.8890, 95.3317, 92.7613
bev  AP:97.9439, 95.6098, 95.0311
3d   AP:97.9191, 95.5267, 94.8690
aos  AP:97.86, 95.24, 92.57
Pedestrian [email protected], 0.50, 0.50:
bbox AP:72.8288, 70.9435, 67.4487
bev  AP:66.6826, 61.2784, 57.2613
3d   AP:60.7116, 58.2224, 52.1716
aos  AP:68.53, 65.94, 62.35
Pedestrian [email protected], 0.50, 0.50:
bbox AP:74.6393, 70.8025, 67.1719
bev  AP:66.3106, 61.6625, 56.5282
3d   AP:61.5620, 56.8584, 51.7222
aos  AP:69.55, 65.18, 61.33
Pedestrian [email protected], 0.25, 0.25:
bbox AP:72.8288, 70.9435, 67.4487
bev  AP:81.5755, 78.6000, 73.2873
3d   AP:81.5028, 78.5548, 73.2312
aos  AP:68.53, 65.94, 62.35
Pedestrian [email protected], 0.25, 0.25:
bbox AP:74.6393, 70.8025, 67.1719
bev  AP:82.6367, 79.7249, 75.3315
3d   AP:82.5666, 79.6892, 75.1957
aos  AP:69.55, 65.18, 61.33
Cyclist [email protected], 0.50, 0.50:
bbox AP:95.4895, 78.1226, 76.4692
bev  AP:93.0930, 74.5530, 72.0119
3d   AP:84.9917, 71.1507, 68.6455
aos  AP:95.39, 77.80, 76.02
Cyclist [email protected], 0.50, 0.50:
bbox AP:96.5594, 81.0552, 78.2527
bev  AP:94.3008, 75.5013, 72.4293
3d   AP:89.7271, 71.4147, 68.1409
aos  AP:96.45, 80.69, 77.77
Cyclist [email protected], 0.25, 0.25:
bbox AP:95.4895, 78.1226, 76.4692
bev  AP:94.3544, 79.1233, 74.0973
3d   AP:94.3544, 79.1233, 74.0973
aos  AP:95.39, 77.80, 76.02
Cyclist [email protected], 0.25, 0.25:
bbox AP:96.5594, 81.0552, 78.2527
bev  AP:95.5326, 79.4494, 75.4784
3d   AP:95.5326, 79.4494, 75.4784
aos  AP:96.45, 80.69, 77.77
    • testing with batch_size=1
    Car [email protected], 0.70, 0.70:
bbox AP:96.4403, 90.1502, 89.4654
bev  AP:90.4357, 88.8551, 87.2133
3d   AP:89.4362, 79.5503, 78.4923
aos  AP:96.39, 90.05, 89.27
Car [email protected], 0.70, 0.70:
bbox AP:98.1592, 95.2860, 92.7122
bev  AP:95.0694, 91.5086, 88.9651
3d   AP:91.9366, 83.4404, 80.4498
aos  AP:98.11, 95.15, 92.49
Car [email protected], 0.50, 0.50:
bbox AP:96.4403, 90.1502, 89.4654
bev  AP:96.5742, 90.2334, 89.7347
3d   AP:96.5268, 90.2056, 89.6426
aos  AP:96.39, 90.05, 89.27
Car [email protected], 0.50, 0.50:
bbox AP:98.1592, 95.2860, 92.7122
bev  AP:98.2310, 95.6126, 95.0153
3d   AP:98.2032, 95.5374, 94.8227
aos  AP:98.11, 95.15, 92.49
Pedestrian [email protected], 0.50, 0.50:
bbox AP:72.4337, 70.8737, 67.2378
bev  AP:67.0958, 61.2871, 57.6957
3d   AP:63.0052, 58.1972, 51.7695
aos  AP:67.72, 65.71, 61.87
Pedestrian [email protected], 0.50, 0.50:
bbox AP:74.4579, 71.7198, 67.1743
bev  AP:66.6856, 61.9405, 56.6848
3d   AP:62.4795, 56.7675, 51.4291
aos  AP:68.90, 65.67, 60.93
Pedestrian [email protected], 0.25, 0.25:
bbox AP:72.4337, 70.8737, 67.2378
bev  AP:82.2410, 79.3855, 73.3600
3d   AP:82.1994, 79.1053, 73.2722
aos  AP:67.72, 65.71, 61.87
Pedestrian [email protected], 0.25, 0.25:
bbox AP:74.4579, 71.7198, 67.1743
bev  AP:84.0556, 80.6710, 75.3752
3d   AP:83.9976, 80.5698, 75.2735
aos  AP:68.90, 65.67, 60.93
Cyclist [email protected], 0.50, 0.50:
bbox AP:95.0359, 77.4640, 76.2851
bev  AP:85.9516, 72.4271, 70.1423
3d   AP:84.7977, 69.2345, 64.6304
aos  AP:94.92, 77.19, 75.87
Cyclist [email protected], 0.50, 0.50:
bbox AP:96.3046, 80.8322, 77.9135
bev  AP:90.6485, 72.6650, 69.5093
3d   AP:89.1079, 69.1793, 64.9591
aos  AP:96.18, 80.47, 77.49
Cyclist [email protected], 0.25, 0.25:
bbox AP:95.0359, 77.4640, 76.2851
bev  AP:93.6113, 74.7370, 73.4226
3d   AP:93.6113, 74.7370, 73.4226
aos  AP:94.92, 77.19, 75.87
Cyclist [email protected], 0.25, 0.25:
bbox AP:96.3046, 80.8322, 77.9135
bev  AP:94.9939, 77.7644, 74.7668
3d   AP:94.9939, 77.7644, 74.7668
aos  AP:96.18, 80.47, 77.49

    Specifically, the AP_0.5 of Cyclist under the batch_size=1 setting is about 1.9 lower than that under the batch_size=8 setting, and the AP_0.25 of Cyclist under the batch_size=1 setting is nearly 5 lower than that under the batch_size=8 setting.

    So why would this happen? Can you give me some insights to handle this problem? Thanks!

    opened by Monster2333333 12
  • About kitti test result

    About kitti test result

    Hi, yifan! Thanks for you wounderful work. I eval your model on kitti val dataset and get the same result as your repo. However, when i submit the test result on kitti website. I can not get the result in your paper. 2022-04-21 00:46:25,841 INFO *************** Performance of EPOCH no_number ***************** 2022-04-21 00:46:25,841 INFO Generate label finished(sec_per_example: 0.0177 second). 2022-04-21 00:46:25,841 INFO recall_roi_0.3: 0.000000 2022-04-21 00:46:25,841 INFO recall_rcnn_0.3: 0.927099 2022-04-21 00:46:25,842 INFO recall_roi_0.5: 0.000000 2022-04-21 00:46:25,842 INFO recall_rcnn_0.5: 0.884041 2022-04-21 00:46:25,842 INFO recall_roi_0.7: 0.000000 2022-04-21 00:46:25,842 INFO recall_rcnn_0.7: 0.663060 2022-04-21 00:46:25,848 INFO Average predicted number of objects(3769 samples): 7.721 2022-04-21 00:46:57,396 INFO Car [email protected], 0.70, 0.70: bbox AP:96.2779, 90.1569, 89.5057 bev AP:90.3980, 88.8603, 86.9664 3d AP:89.3976, 79.5625, 78.4435 aos AP:96.25, 90.10, 89.35 Car [email protected], 0.70, 0.70: bbox AP:97.9059, 95.3214, 92.7594 bev AP:94.7520, 91.3533, 88.8415 3d AP:91.7990, 83.3737, 80.3455 aos AP:97.88, 95.23, 92.57 Car [email protected], 0.50, 0.50: bbox AP:96.2779, 90.1569, 89.5057 bev AP:96.3642, 90.2276, 89.7638 3d AP:96.3233, 90.2013, 89.6963 aos AP:96.25, 90.10, 89.35 Car [email protected], 0.50, 0.50: bbox AP:97.9059, 95.3214, 92.7594 bev AP:97.9552, 95.6085, 95.0305 3d AP:97.9302, 95.5291, 94.8719 aos AP:97.88, 95.23, 92.57 Pedestrian [email protected], 0.50, 0.50: bbox AP:72.7975, 70.9381, 67.4165 bev AP:66.7870, 61.3725, 57.6589 3d AP:60.8069, 58.3171, 52.2580 aos AP:68.50, 65.93, 62.32 Pedestrian [email protected], 0.50, 0.50: bbox AP:74.5996, 70.7681, 67.1326 bev AP:66.4324, 61.7818, 56.7088 3d AP:61.6833, 56.9830, 51.8147 aos AP:69.51, 65.15, 61.29 Pedestrian [email protected], 0.25, 0.25: bbox AP:72.7975, 70.9381, 67.4165 bev AP:81.5397, 78.5586, 73.2419 3d AP:81.4691, 78.5135, 73.1643 aos AP:68.50, 65.93, 62.32 Pedestrian [email protected], 0.25, 0.25: bbox AP:74.5996, 70.7681, 67.1326 bev AP:82.5906, 79.6797, 75.2987 3d AP:82.5212, 79.6466, 75.1587 aos AP:69.51, 65.15, 61.29 Cyclist [email protected], 0.50, 0.50: bbox AP:95.5281, 78.1780, 76.5017 bev AP:93.1435, 74.5972, 72.0342 3d AP:85.1286, 71.1990, 68.6697 aos AP:95.43, 77.85, 76.05 Cyclist [email protected], 0.50, 0.50: bbox AP:96.6252, 81.1220, 78.3019 bev AP:94.3767, 75.5814, 72.4732 3d AP:89.8377, 71.4524, 68.1976 aos AP:96.52, 80.74, 77.81 Cyclist [email protected], 0.25, 0.25: bbox AP:95.5281, 78.1780, 76.5017 bev AP:94.4477, 79.1653, 74.1242 3d AP:94.4477, 79.1653, 74.1242 aos AP:95.43, 77.85, 76.05 Cyclist [email protected], 0.25, 0.25: bbox AP:96.6252, 81.1220, 78.3019 bev AP:95.6348, 79.5295, 75.5297 3d AP:95.6348, 79.5295, 75.5297 aos AP:96.52, 80.74, 77.81

    image

    opened by neineimeow 7
  • The running results are higher than those in the paper

    The running results are higher than those in the paper

    Here are my results：

    Car [email protected], 0.70, 0.70: bbox AP:96.4483, 90.1338, 89.4304 bev AP:90.4018, 88.7234, 86.8532 3d AP:89.2549, 79.5031, 78.3933 aos AP:96.41, 90.06, 89.26 Car [email protected], 0.70, 0.70: bbox AP:98.0873, 95.2228, 92.6929 bev AP:94.9569, 89.7264, 88.7445 3d AP:91.5120, 83.3311, 80.3146 aos AP:98.06, 95.13, 92.50 Car [email protected], 0.50, 0.50: bbox AP:96.4483, 90.1338, 89.4304 bev AP:96.5648, 90.2485, 89.7516 3d AP:96.5033, 90.2212, 89.6900 aos AP:96.41, 90.06, 89.26 Car [email protected], 0.50, 0.50: bbox AP:98.0873, 95.2228, 92.6929 bev AP:98.1673, 95.6244, 95.0024 3d AP:98.1293, 95.5496, 94.8515 aos AP:98.06, 95.13, 92.50 Pedestrian [email protected], 0.50, 0.50: bbox AP:72.6155, 70.4817, 67.0930 bev AP:67.1150, 61.2996, 56.7766 3d AP:63.1800, 58.5233, 52.3548 aos AP:68.11, 65.19, 61.52 Pedestrian [email protected], 0.50, 0.50: bbox AP:74.2353, 70.4244, 66.7236 bev AP:66.8335, 61.8515, 56.3665 3d AP:62.7567, 57.2351, 51.7856 aos AP:69.12, 64.52, 60.56 Pedestrian [email protected], 0.25, 0.25: bbox AP:72.6155, 70.4817, 67.0930 bev AP:81.8792, 78.3813, 73.3151 3d AP:81.6713, 78.2371, 73.1219 aos AP:68.11, 65.19, 61.52 Pedestrian [email protected], 0.25, 0.25: bbox AP:74.2353, 70.4244, 66.7236 bev AP:83.0304, 79.7448, 75.3895 3d AP:82.8435, 79.5333, 75.1033 aos AP:69.12, 64.52, 60.56 Cyclist [email protected], 0.50, 0.50: bbox AP:95.2029, 81.5999, 76.7175 bev AP:93.4303, 74.6626, 71.6172 3d AP:92.6345, 73.2495, 70.1991 aos AP:95.11, 81.35, 76.46 Cyclist [email protected], 0.50, 0.50: bbox AP:96.5793, 82.3327, 78.2697 bev AP:94.9052, 76.9932, 72.6225 3d AP:93.8756, 74.3714, 69.8898 aos AP:96.48, 82.08, 77.98 Cyclist [email protected], 0.25, 0.25: bbox AP:95.2029, 81.5999, 76.7175 bev AP:94.0906, 79.0643, 74.1753 3d AP:94.0766, 79.0527, 74.1413 aos AP:95.11, 81.35, 76.46 Cyclist [email protected], 0.25, 0.25: bbox AP:96.5793, 82.3327, 78.2697 bev AP:95.5894, 79.8844, 75.5143 3d AP:95.5789, 79.8644, 75.3781 aos AP:96.48, 82.08, 77.98

    I understand that this is the result on KITTI data set: 83.3311 57.2351 74.3714. The result given in the paper is the result on the test set: 80.13 39.03 61.94

    Can you tell me why there is such a big gap

    opened by hhhmrcscs 5
  • Question about centroid prediction loss

    Question about centroid prediction loss

    Hi! I just read about this paper, IA-SSD's performance is quite amazing! But I got some trouble when trying to understand the centroid predction loss image Here are the questions:

    1. What do the F+ and j mean in the formula?
    2. How to understand the second term |c_ij - c_i|? Quating the paper: to minimize the uncertainty of the centroid prediction Thanks in advance!
    opened by DeclK 5
  • Question about the GIF figure shown in README

    Question about the GIF figure shown in README

    How to get a series of continuous point cloud in the KITTI dataset rather than the shuffled one like what in the training and testing sets. many thanks!

    opened by OuyangJunyuan 4
  • RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.

    RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.

    Dear， A RuntimeError occurred while I was training on the Kitti dataset, as follows

    Traceback (most recent call last):
  File "/media/jiangtao/Project/IA-SSD-main/tools/train_utils/train_utils.py", line 52, in train_one_epoch
    loss.backward()

  File "/home/zhang/anaconda3/envs/second1/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)

  File "/home/zhang/anaconda3/envs/second1/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag

RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

    Could you please give me the suggestions？ Looking forward to your reply.

    my environment Ubuntu18.04 Python = 3.6 PyTorch = 1.4 CUDA = 10.2 spconv = 1.2.1

    opened by jiangtao129 3
  • about kitti video

    about kitti video

    Hi, this is a wonderful work. I found in your readme.md you show Qualitative results of our IA-SSD on KITTI dataset. How you get kitti video? Is it kitti 360 dataset?

    opened by mc171819 3
  • AssertionError

    AssertionError

    作者你好，我在训练IASSD时到68%，使用给的权重也在测试到68%报错，报错如下希望楼主解答非常感谢。 Traceback (most recent call last): File "test.py", line 203, in main() File "test.py", line 199, in main eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id, dist_test=dist_test) File "test.py", line 65, in eval_single_ckpt eval_utils.eval_one_epoch( File "/data3/huangmang2/IA-SSD/tools/eval_utils/eval_utils.py", line 54, in eval_one_epoch for i, batch_dict in enumerate(dataloader): File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in next data = self._next_data() File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data return self._process_data(data) File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data data.reraise() File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) AssertionError: Caught AssertionError in DataLoader worker process 2. Original Traceback (most recent call last): File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop data = fetcher.fetch(index) File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/data3/huangmang2/IA-SSD/tools/../pcdet/datasets/kitti/kitti_dataset.py", line 408, in getitem points = self.get_lidar(sample_idx) File "/data3/huangmang2/IA-SSD/tools/../pcdet/datasets/kitti/kitti_dataset.py", line 65, in get_lidar assert lidar_file.exists() AssertionError

    opened by huangmang 2
  • Could you release the code of calculating the instance recall rate?

    Could you release the code of calculating the instance recall rate?

    @QingyongHu @yifanzhang713 Hi, thanks for your work. The following is the results I reproduced, but it is much different(car class) from the results in the paper. Could you release the code of calculating the instance recall rate for foreground points ?

                           4096 points          ...  256 points
in paper    Ctr-aware  98.3%  100%   97.2%  ...  97.9%  98.4%  97.2%
mine        Ctr-aware  93.4%  99.6%  98.3%  ...  92.2%  97.3%  95.2%
    opened by kellen5l 2
  • about gpu memory

    about gpu memory

    Hi, I also wonder the memory in your table6. I set batchsize=1 when inference in V100, but I get the memory is 1800+M, how can i get the 102M or near 120M result?

    opened by mc171819 2
  • SyntaxError: invalid syntax

    SyntaxError: invalid syntax

    您好：请问出现SyntaxError: invalid syntax是什么原因？

    $ python test.py from tifffile import TiffFile File "/home/fd/anaconda3/envs/IASSD/lib/python3.7/site-packages/tifffile-2022.10.10-py3.7.egg/tifffile/init.py", line 3, in from .tifffile import doc, all, version, main File "/home/fd/anaconda3/envs/IASSD/lib/python3.7/site-packages/tifffile-2022.10.10-py3.7.egg/tifffile/tifffile.py", line 1083 /, ^ SyntaxError: invalid syntax

    opened by 346575116 0
  • Lower Performance of PVRCNN on Waymo

    Lower Performance of PVRCNN on Waymo

    Exciting work! I wonder why the performance of PVCNN on Waymo validation set is lower than that of the original paper. I also noticed that the performance of other methods in Table 4 are also lower than the original paper.

    opened by JingweiZhang12 0
  • Question regarding efficiency

    Question regarding efficiency

    Hi, thank you for the interesting work! I was wondering - what is the main source of the FPS difference between IA-SSD and 3DSSD? Looking at the general model flow of both methods, they seemed to have similar backbones with similar # of points. However, the FPS difference is 83 vs 28. I was wondering if you had any insight as to why.

    opened by Divadi 1
  • Inference speed gap in waymo dataset

    Inference speed gap in waymo dataset

    Thank you for your work, I used a single A40 to test the IA-SSD in the waymo dataset and found that the inference speed was only 2.2FPS, while in the paper this data was 8FPS, how should I set the parameters to improve the inference speed .

    opened by hht1996ok 0
  • How to generate GIF images?

    How to generate GIF images?

    Thank you for the amazing work! Your result in GIF format is very interesting and intuitive.

    I want to export the result in GIF format as you uploaded in docs. Do you mind if I ask you to provide the python code to generate the GIF?

    Best, Dongmin Choi.

    opened by ChoiDM 0
Owner
Yifan Zhang
Yifan Zhang
GitHub
(CVPR 2022 Oral) Official implementation for "Surface Representation for Point Clouds"

RepSurf - Surface Representation for Point Clouds [CVPR 2022 Oral] By Haoxi Ran* , Jun Liu, Chengjie Wang ( * : corresponding contact) The pytorch off

Haoxi Ran 264 Dec 23, 2022
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

BRNet Introduction This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds,

null 86 Oct 5, 2022
Implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

SemCo The official pytorch implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

null 42 Nov 14, 2022
[CVPR 2022 Oral] EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

EPro-PnP EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation In CVPR 2022 (Oral). [paper] Hanshen

同济大学智能汽车研究所综合感知研究组 ( Comprehensive Perception Research Group under Institute of Intelligent Vehicles, School of Automotive Studies, Tongji University) 842 Jan 4, 2023
Scribble-Supervised LiDAR Semantic Segmentation, CVPR 2022 (ORAL)

Scribble-Supervised LiDAR Semantic Segmentation Dataset and code release for the paper Scribble-Supervised LiDAR Semantic Segmentation, CVPR 2022 (ORA

null 102 Dec 25, 2022
A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning.

Open3DSOT A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning. The official code release of BAT an

Kangel Zenn 172 Dec 23, 2022
Implementation of CVPR'2022:Surface Reconstruction from Point Clouds by Learning Predictive Context Priors

Surface Reconstruction from Point Clouds by Learning Predictive Context Priors (CVPR 2022) Personal Web Pages | Paper | Project Page This repository c

null 136 Dec 12, 2022
Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds (ICCV 2021 oral) **Project Page | Arxiv ** Runsong Zhu¹, Yuan Liu², Zhen Dong¹, Te

null 40 Dec 30, 2022
Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving Abstract In this paper, we introduce SalsaNext f

null 308 Jan 4, 2023
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR fog simulation Created by Martin Hahner at the Computer Vision Lab of ETH Zurich. This is the official code release of the paper Fog Simulation

Martin Hahner 110 Dec 30, 2022
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR2022)[paper] Authors: Chenhang He, Ruihuang Li, Shuai Li, L

Billy HE 141 Dec 30, 2022
Implementation of CVPR'2022:Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors

Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors (CVPR 2022) Personal Web Pages | Paper | Project Page This repository contains

null 151 Dec 26, 2022
Erpnext app for make employee salary on payroll entry based on one or more project with percentage for all project equal 100 %

Project Payroll this app for make payroll for employee based on projects like project on 30 % and project 2 70 % as account dimension it makes genral

Ibrahim Morghim 8 Jan 2, 2023
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 5, 2022
[CVPR 2022] TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing

TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing (CVPR 2022) This repository provides the official PyTorch impleme

Billy XU 128 Jan 3, 2023
The code for our paper submitted to RAL/IROS 2022: OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition.

OverlapTransformer The code for our paper submitted to RAL/IROS 2022: OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for

HAOMO.AI 136 Jan 3, 2023
Temporally Efficient Vision Transformer for Video Instance Segmentation, CVPR 2022, Oral

Temporally Efficient Vision Transformer for Video Instance Segmentation Temporally Efficient Vision Transformer for Video Instance Segmentation (CVPR

Hust Visual Learning Team 203 Dec 31, 2022
Attention-based Transformation from Latent Features to Point Clouds (AAAI 2022)

Attention-based Transformation from Latent Features to Point Clouds This repository contains a PyTorch implementation of the paper: Attention-based Tr

null 12 Nov 11, 2022
《Train in Germany, Test in The USA: Making 3D Object Detectors Generalize》(CVPR 2020)

Train in Germany, Test in The USA: Making 3D Object Detectors Generalize This paper has been accpeted by Conference on Computer Vision and Pattern Rec

Xiangyu Chen 101 Jan 2, 2023