The same frame point cloud is inputed into the model, but the result is different each time. For example. for the same frame point cloud, two cars are detected by the model for the first time, but three cars are detected by the model for the second time.The results are very erratic. Can you explain that, please?

It seems that the batch size will affect the permformance when testing the model using the same pretrained weight file. Here's my result:

• testing with batch_size=8

Car [email protected], 0.70, 0.70:
bbox AP:96.2771, 90.1569, 89.5080
bev  AP:90.4018, 88.8616, 86.9882
3d   AP:89.3976, 79.5647, 78.4486
aos  AP:96.25, 90.10, 89.35
Car [email protected], 0.70, 0.70:
bbox AP:97.8890, 95.3317, 92.7613
bev  AP:94.7081, 91.3551, 88.8493
3d   AP:91.7842, 83.3762, 80.3495
aos  AP:97.86, 95.24, 92.57
Car [email protected], 0.50, 0.50:
bbox AP:96.2771, 90.1569, 89.5080
bev  AP:96.3634, 90.2276, 89.7638
3d   AP:96.3225, 90.2001, 89.6938
aos  AP:96.25, 90.10, 89.35
Car [email protected], 0.50, 0.50:
bbox AP:97.8890, 95.3317, 92.7613
bev  AP:97.9439, 95.6098, 95.0311
3d   AP:97.9191, 95.5267, 94.8690
aos  AP:97.86, 95.24, 92.57
Pedestrian [email protected], 0.50, 0.50:
bbox AP:72.8288, 70.9435, 67.4487
bev  AP:66.6826, 61.2784, 57.2613
3d   AP:60.7116, 58.2224, 52.1716
aos  AP:68.53, 65.94, 62.35
Pedestrian [email protected], 0.50, 0.50:
bbox AP:74.6393, 70.8025, 67.1719
bev  AP:66.3106, 61.6625, 56.5282
3d   AP:61.5620, 56.8584, 51.7222
aos  AP:69.55, 65.18, 61.33
Pedestrian [email protected], 0.25, 0.25:
bbox AP:72.8288, 70.9435, 67.4487
bev  AP:81.5755, 78.6000, 73.2873
3d   AP:81.5028, 78.5548, 73.2312
aos  AP:68.53, 65.94, 62.35
Pedestrian [email protected], 0.25, 0.25:
bbox AP:74.6393, 70.8025, 67.1719
bev  AP:82.6367, 79.7249, 75.3315
3d   AP:82.5666, 79.6892, 75.1957
aos  AP:69.55, 65.18, 61.33
Cyclist [email protected], 0.50, 0.50:
bbox AP:95.4895, 78.1226, 76.4692
bev  AP:93.0930, 74.5530, 72.0119
3d   AP:84.9917, 71.1507, 68.6455
aos  AP:95.39, 77.80, 76.02
Cyclist [email protected], 0.50, 0.50:
bbox AP:96.5594, 81.0552, 78.2527
bev  AP:94.3008, 75.5013, 72.4293
3d   AP:89.7271, 71.4147, 68.1409
aos  AP:96.45, 80.69, 77.77
Cyclist [email protected], 0.25, 0.25:
bbox AP:95.4895, 78.1226, 76.4692
bev  AP:94.3544, 79.1233, 74.0973
3d   AP:94.3544, 79.1233, 74.0973
aos  AP:95.39, 77.80, 76.02
Cyclist [email protected], 0.25, 0.25:
bbox AP:96.5594, 81.0552, 78.2527
bev  AP:95.5326, 79.4494, 75.4784
3d   AP:95.5326, 79.4494, 75.4784
aos  AP:96.45, 80.69, 77.77

• testing with batch_size=1

Car [email protected], 0.70, 0.70:
bbox AP:96.4403, 90.1502, 89.4654
bev  AP:90.4357, 88.8551, 87.2133
3d   AP:89.4362, 79.5503, 78.4923
aos  AP:96.39, 90.05, 89.27
Car [email protected], 0.70, 0.70:
bbox AP:98.1592, 95.2860, 92.7122
bev  AP:95.0694, 91.5086, 88.9651
3d   AP:91.9366, 83.4404, 80.4498
aos  AP:98.11, 95.15, 92.49
Car [email protected], 0.50, 0.50:
bbox AP:96.4403, 90.1502, 89.4654
bev  AP:96.5742, 90.2334, 89.7347
3d   AP:96.5268, 90.2056, 89.6426
aos  AP:96.39, 90.05, 89.27
Car [email protected], 0.50, 0.50:
bbox AP:98.1592, 95.2860, 92.7122
bev  AP:98.2310, 95.6126, 95.0153
3d   AP:98.2032, 95.5374, 94.8227
aos  AP:98.11, 95.15, 92.49
Pedestrian [email protected], 0.50, 0.50:
bbox AP:72.4337, 70.8737, 67.2378
bev  AP:67.0958, 61.2871, 57.6957
3d   AP:63.0052, 58.1972, 51.7695
aos  AP:67.72, 65.71, 61.87
Pedestrian [email protected], 0.50, 0.50:
bbox AP:74.4579, 71.7198, 67.1743
bev  AP:66.6856, 61.9405, 56.6848
3d   AP:62.4795, 56.7675, 51.4291
aos  AP:68.90, 65.67, 60.93
Pedestrian [email protected], 0.25, 0.25:
bbox AP:72.4337, 70.8737, 67.2378
bev  AP:82.2410, 79.3855, 73.3600
3d   AP:82.1994, 79.1053, 73.2722
aos  AP:67.72, 65.71, 61.87
Pedestrian [email protected], 0.25, 0.25:
bbox AP:74.4579, 71.7198, 67.1743
bev  AP:84.0556, 80.6710, 75.3752
3d   AP:83.9976, 80.5698, 75.2735
aos  AP:68.90, 65.67, 60.93
Cyclist [email protected], 0.50, 0.50:
bbox AP:95.0359, 77.4640, 76.2851
bev  AP:85.9516, 72.4271, 70.1423
3d   AP:84.7977, 69.2345, 64.6304
aos  AP:94.92, 77.19, 75.87
Cyclist [email protected], 0.50, 0.50:
bbox AP:96.3046, 80.8322, 77.9135
bev  AP:90.6485, 72.6650, 69.5093
3d   AP:89.1079, 69.1793, 64.9591
aos  AP:96.18, 80.47, 77.49
Cyclist [email protected], 0.25, 0.25:
bbox AP:95.0359, 77.4640, 76.2851
bev  AP:93.6113, 74.7370, 73.4226
3d   AP:93.6113, 74.7370, 73.4226
aos  AP:94.92, 77.19, 75.87
Cyclist [email protected], 0.25, 0.25:
bbox AP:96.3046, 80.8322, 77.9135
bev  AP:94.9939, 77.7644, 74.7668
3d   AP:94.9939, 77.7644, 74.7668
aos  AP:96.18, 80.47, 77.49

Specifically, the AP_0.5 of Cyclist under the batch_size=1 setting is about 1.9 lower than that under the batch_size=8 setting, and the AP_0.25 of Cyclist under the batch_size=1 setting is nearly 5 lower than that under the batch_size=8 setting.

So why would this happen? Can you give me some insights to handle this problem? Thanks!

Hi, yifan! Thanks for you wounderful work. I eval your model on kitti val dataset and get the same result as your repo. However, when i submit the test result on kitti website. I can not get the result in your paper. 2022-04-21 00:46:25,841 INFO *************** Performance of EPOCH no_number ***************** 2022-04-21 00:46:25,841 INFO Generate label finished(sec_per_example: 0.0177 second). 2022-04-21 00:46:25,841 INFO recall_roi_0.3: 0.000000 2022-04-21 00:46:25,841 INFO recall_rcnn_0.3: 0.927099 2022-04-21 00:46:25,842 INFO recall_roi_0.5: 0.000000 2022-04-21 00:46:25,842 INFO recall_rcnn_0.5: 0.884041 2022-04-21 00:46:25,842 INFO recall_roi_0.7: 0.000000 2022-04-21 00:46:25,842 INFO recall_rcnn_0.7: 0.663060 2022-04-21 00:46:25,848 INFO Average predicted number of objects(3769 samples): 7.721 2022-04-21 00:46:57,396 INFO Car [email protected], 0.70, 0.70: bbox AP:96.2779, 90.1569, 89.5057 bev AP:90.3980, 88.8603, 86.9664 3d AP:89.3976, 79.5625, 78.4435 aos AP:96.25, 90.10, 89.35 Car [email protected], 0.70, 0.70: bbox AP:97.9059, 95.3214, 92.7594 bev AP:94.7520, 91.3533, 88.8415 3d AP:91.7990, 83.3737, 80.3455 aos AP:97.88, 95.23, 92.57 Car [email protected], 0.50, 0.50: bbox AP:96.2779, 90.1569, 89.5057 bev AP:96.3642, 90.2276, 89.7638 3d AP:96.3233, 90.2013, 89.6963 aos AP:96.25, 90.10, 89.35 Car [email protected], 0.50, 0.50: bbox AP:97.9059, 95.3214, 92.7594 bev AP:97.9552, 95.6085, 95.0305 3d AP:97.9302, 95.5291, 94.8719 aos AP:97.88, 95.23, 92.57 Pedestrian [email protected], 0.50, 0.50: bbox AP:72.7975, 70.9381, 67.4165 bev AP:66.7870, 61.3725, 57.6589 3d AP:60.8069, 58.3171, 52.2580 aos AP:68.50, 65.93, 62.32 Pedestrian [email protected], 0.50, 0.50: bbox AP:74.5996, 70.7681, 67.1326 bev AP:66.4324, 61.7818, 56.7088 3d AP:61.6833, 56.9830, 51.8147 aos AP:69.51, 65.15, 61.29 Pedestrian [email protected], 0.25, 0.25: bbox AP:72.7975, 70.9381, 67.4165 bev AP:81.5397, 78.5586, 73.2419 3d AP:81.4691, 78.5135, 73.1643 aos AP:68.50, 65.93, 62.32 Pedestrian [email protected], 0.25, 0.25: bbox AP:74.5996, 70.7681, 67.1326 bev AP:82.5906, 79.6797, 75.2987 3d AP:82.5212, 79.6466, 75.1587 aos AP:69.51, 65.15, 61.29 Cyclist [email protected], 0.50, 0.50: bbox AP:95.5281, 78.1780, 76.5017 bev AP:93.1435, 74.5972, 72.0342 3d AP:85.1286, 71.1990, 68.6697 aos AP:95.43, 77.85, 76.05 Cyclist [email protected], 0.50, 0.50: bbox AP:96.6252, 81.1220, 78.3019 bev AP:94.3767, 75.5814, 72.4732 3d AP:89.8377, 71.4524, 68.1976 aos AP:96.52, 80.74, 77.81 Cyclist [email protected], 0.25, 0.25: bbox AP:95.5281, 78.1780, 76.5017 bev AP:94.4477, 79.1653, 74.1242 3d AP:94.4477, 79.1653, 74.1242 aos AP:95.43, 77.85, 76.05 Cyclist [email protected], 0.25, 0.25: bbox AP:96.6252, 81.1220, 78.3019 bev AP:95.6348, 79.5295, 75.5297 3d AP:95.6348, 79.5295, 75.5297 aos AP:96.52, 80.74, 77.81

Here are my results：

Car [email protected], 0.70, 0.70: bbox AP:96.4483, 90.1338, 89.4304 bev AP:90.4018, 88.7234, 86.8532 3d AP:89.2549, 79.5031, 78.3933 aos AP:96.41, 90.06, 89.26 Car [email protected], 0.70, 0.70: bbox AP:98.0873, 95.2228, 92.6929 bev AP:94.9569, 89.7264, 88.7445 3d AP:91.5120, 83.3311, 80.3146 aos AP:98.06, 95.13, 92.50 Car [email protected], 0.50, 0.50: bbox AP:96.4483, 90.1338, 89.4304 bev AP:96.5648, 90.2485, 89.7516 3d AP:96.5033, 90.2212, 89.6900 aos AP:96.41, 90.06, 89.26 Car [email protected], 0.50, 0.50: bbox AP:98.0873, 95.2228, 92.6929 bev AP:98.1673, 95.6244, 95.0024 3d AP:98.1293, 95.5496, 94.8515 aos AP:98.06, 95.13, 92.50 Pedestrian [email protected], 0.50, 0.50: bbox AP:72.6155, 70.4817, 67.0930 bev AP:67.1150, 61.2996, 56.7766 3d AP:63.1800, 58.5233, 52.3548 aos AP:68.11, 65.19, 61.52 Pedestrian [email protected], 0.50, 0.50: bbox AP:74.2353, 70.4244, 66.7236 bev AP:66.8335, 61.8515, 56.3665 3d AP:62.7567, 57.2351, 51.7856 aos AP:69.12, 64.52, 60.56 Pedestrian [email protected], 0.25, 0.25: bbox AP:72.6155, 70.4817, 67.0930 bev AP:81.8792, 78.3813, 73.3151 3d AP:81.6713, 78.2371, 73.1219 aos AP:68.11, 65.19, 61.52 Pedestrian [email protected], 0.25, 0.25: bbox AP:74.2353, 70.4244, 66.7236 bev AP:83.0304, 79.7448, 75.3895 3d AP:82.8435, 79.5333, 75.1033 aos AP:69.12, 64.52, 60.56 Cyclist [email protected], 0.50, 0.50: bbox AP:95.2029, 81.5999, 76.7175 bev AP:93.4303, 74.6626, 71.6172 3d AP:92.6345, 73.2495, 70.1991 aos AP:95.11, 81.35, 76.46 Cyclist [email protected], 0.50, 0.50: bbox AP:96.5793, 82.3327, 78.2697 bev AP:94.9052, 76.9932, 72.6225 3d AP:93.8756, 74.3714, 69.8898 aos AP:96.48, 82.08, 77.98 Cyclist [email protected], 0.25, 0.25: bbox AP:95.2029, 81.5999, 76.7175 bev AP:94.0906, 79.0643, 74.1753 3d AP:94.0766, 79.0527, 74.1413 aos AP:95.11, 81.35, 76.46 Cyclist [email protected], 0.25, 0.25: bbox AP:96.5793, 82.3327, 78.2697 bev AP:95.5894, 79.8844, 75.5143 3d AP:95.5789, 79.8644, 75.3781 aos AP:96.48, 82.08, 77.98

I understand that this is the result on KITTI data set: 83.3311 57.2351 74.3714. The result given in the paper is the result on the test set: 80.13 39.03 61.94

Can you tell me why there is such a big gap

Hi! I just read about this paper, IA-SSD's performance is quite amazing! But I got some trouble when trying to understand the centroid predction loss Here are the questions:

1. What do the F+ and j mean in the formula?
2. How to understand the second term |c_ij - c_i|? Quating the paper: to minimize the uncertainty of the centroid prediction Thanks in advance!

How to get a series of continuous point cloud in the KITTI dataset rather than the shuffled one like what in the training and testing sets. many thanks!

Dear， A RuntimeError occurred while I was training on the Kitti dataset, as follows

Traceback (most recent call last):
  File "/media/jiangtao/Project/IA-SSD-main/tools/train_utils/train_utils.py", line 52, in train_one_epoch
    loss.backward()

  File "/home/zhang/anaconda3/envs/second1/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)

  File "/home/zhang/anaconda3/envs/second1/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag

RuntimeError: Expected isFloatingType(grads[i].type().scalarType()) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

Could you please give me the suggestions？ Looking forward to your reply.

my environment Ubuntu18.04 Python = 3.6 PyTorch = 1.4 CUDA = 10.2 spconv = 1.2.1

Hi, this is a wonderful work. I found in your readme.md you show Qualitative results of our IA-SSD on KITTI dataset. How you get kitti video? Is it kitti 360 dataset?

作者你好，我在训练IASSD时到68%，使用给的权重也在测试到68%报错，报错如下希望楼主解答非常感谢。 Traceback (most recent call last): File "test.py", line 203, in main() File "test.py", line 199, in main eval_single_ckpt(model, test_loader, args, eval_output_dir, logger, epoch_id, dist_test=dist_test) File "test.py", line 65, in eval_single_ckpt eval_utils.eval_one_epoch( File "/data3/huangmang2/IA-SSD/tools/eval_utils/eval_utils.py", line 54, in eval_one_epoch for i, batch_dict in enumerate(dataloader): File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in next data = self._next_data() File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data return self._process_data(data) File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data data.reraise() File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg) AssertionError: Caught AssertionError in DataLoader worker process 2. Original Traceback (most recent call last): File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop data = fetcher.fetch(index) File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/data2/huangmang/anaconda3/envs/IASSD/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/data3/huangmang2/IA-SSD/tools/../pcdet/datasets/kitti/kitti_dataset.py", line 408, in getitem points = self.get_lidar(sample_idx) File "/data3/huangmang2/IA-SSD/tools/../pcdet/datasets/kitti/kitti_dataset.py", line 65, in get_lidar assert lidar_file.exists() AssertionError

@QingyongHu @yifanzhang713 Hi, thanks for your work. The following is the results I reproduced, but it is much different(car class) from the results in the paper. Could you release the code of calculating the instance recall rate for foreground points ?

                       4096 points          ...  256 points
in paper    Ctr-aware  98.3%  100%   97.2%  ...  97.9%  98.4%  97.2%
mine        Ctr-aware  93.4%  99.6%  98.3%  ...  92.2%  97.3%  95.2%

Hi, I also wonder the memory in your table6. I set batchsize=1 when inference in V100, but I get the memory is 1800+M, how can i get the 102M or near 120M result?

您好：请问出现SyntaxError: invalid syntax是什么原因？

$ python test.py from tifffile import TiffFile File "/home/fd/anaconda3/envs/IASSD/lib/python3.7/site-packages/tifffile-2022.10.10-py3.7.egg/tifffile/init.py", line 3, in from .tifffile import doc, all, version, main File "/home/fd/anaconda3/envs/IASSD/lib/python3.7/site-packages/tifffile-2022.10.10-py3.7.egg/tifffile/tifffile.py", line 1083 /, ^ SyntaxError: invalid syntax

Exciting work! I wonder why the performance of PVCNN on Waymo validation set is lower than that of the original paper. I also noticed that the performance of other methods in Table 4 are also lower than the original paper.

Hi, thank you for the interesting work! I was wondering - what is the main source of the FPS difference between IA-SSD and 3DSSD? Looking at the general model flow of both methods, they seemed to have similar backbones with similar # of points. However, the FPS difference is 83 vs 28. I was wondering if you had any insight as to why.

Thank you for your work, I used a single A40 to test the IA-SSD in the waymo dataset and found that the inference speed was only 2.2FPS, while in the paper this data was 8FPS, how should I set the parameters to improve the inference speed .

Thank you for the amazing work! Your result in GIF format is very interesting and intuitive.

I want to export the result in GIF format as you uploaded in docs. Do you mind if I ask you to provide the python code to generate the GIF?

Best, Dongmin Choi.