Hi! Thank you for your work. I'm looking at the code and find that lidar points' features are not exactly aligned: virtual_points1 = data_dict['real_points'][:, [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]] virtual_points2 = data_dict['virtual_points'] points = np.concatenate([points, np.ones([points.shape[0], 15-num_point_feature])], axis=1) virtual_points1 = np.concatenate([virtual_points1, np.zeros([virtual_points1.shape[0], 1])], axis=1) virtual_points2 = np.concatenate([virtual_points2, -1 * np.ones([virtual_points2.shape[0], 1])], axis=1) points = np.concatenate([points, virtual_points1, virtual_points2], axis=0).astype(np.float32)

So I wondered why not using orders like below (x, y, z, intensity(-1 for virtual points2), classification*10, score): virtual_points1 = data_dict['real_points'][:, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]] virtual_points2 = data_dict['virtual_points'] virtual_points2 = np.concatenate([virtual_points2[:, [0, 1, 2]] , -1 * np.ones([virtual_points2.shape[0], 1]), virtual_points2[:, [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]]], axis=1)

Hi! Thanks for your paper MVP. But I have a question about the result of CenterPoint+PointPainting on nuScenes Val dataset. As you mentioned in the paper, it reaches 66.5 overall mAP. But my result is only 65.4. I train the model by just throwing away those virtual points since you have already painted those real points on the masks. All other configurations are the same with https://github.com/tianweiy/CenterPoint/blob/master/configs/mvp/nusc_centerpoint_voxelnet_0075voxel_fix_bn_z_scale_virtual.py

Could you please tell me the reason? Thanks!

I have installed Detectron2 , CenterPoint and CenterNet2, But while running virtual_gen.py in the mvs root folder I get this error "No module named 'CenterNet2'.

I have already added the paths of CenterPoint and CenterNet2.

Hi! Thank you for your work!

1. I'm trying to reproduce the results of MVP-KITTI in your paper, could you provide more information about the pretrained MaskFormer model? Which one should I choose to get the same result as you did.
2. Also, there are some slight differences in results trained with PointPillars and KITTI. I used PointPillars in CenterPoint-KITTI which has the same config file as the one in your OpenPCdet. All settings are default. Also I download the pretrained model in https://github.com/tianweiy/OpenPCDet (model-18M) and get the results as below. The download model gets the same results as documented in your web but I record the Car_R40_moderate_0.7 instead of Car_moderate_0.7 so I can compare it with my result.

Method | Car@R40_moderate | Cyclist@R40_moderate | Pedestrian@R40_moderate PointPillars(my training) | 77.4143 | 64.2696 | 50.4214 PointPillars(download) | 78.3964 | 62.8074 | 51.4145 PointPillars(in paper) | 77.3 | 62.7 | 52.3

How can I get the results in your paper? Are there any changes I need to make?

Thanks for the wonderful job! However, I encountered this error when I tried to generate virtual points:

[04/12 15:03:46 detectron2]: Full config saved to ./output/CenterNet2/nuImages_CenterNet2_DLA_640_8x/config.yaml [04/12 15:03:46 d2.utils.env]: Using a generated random seed 46522022 Loading pretrained [04/12 15:03:55 fvcore.common.checkpoint]: [Checkpointer] Loading from centernet2_checkpoint.pth ... 0%| | 0/2340 [00:00<?, ?it/s]/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [96,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [97,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [98,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [99,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [0,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [1,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [2,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [3,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [4,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [5,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [6,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [7,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [10,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [11,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [12,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [13,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [14,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [15,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [16,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [17,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [18,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [19,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [24,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [25,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [26,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [27,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [28,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [29,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [32,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [33,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [34,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [35,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [36,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [37,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [38,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [39,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [40,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [41,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [42,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [43,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [44,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [45,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [46,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [47,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [48,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [49,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [50,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [51,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [52,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [53,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [54,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [55,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [56,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [57,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [62,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [63,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [64,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [65,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [66,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [67,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [68,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [69,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [70,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [71,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [72,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [73,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [74,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [75,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [76,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [77,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [78,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [79,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [80,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [81,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [82,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [83,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [84,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [85,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [86,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [87,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [88,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [89,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [90,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [91,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [92,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [93,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [94,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. /pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:142: operator(): block: [0,0,0], thread: [95,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed. 0%| | 1/2340 [00:02<1:38:43, 2.53s/it] Traceback (most recent call last): File "virtual_gen.py", line 311, in main(args) File "virtual_gen.py", line 271, in main res = process_one_frame(info, predictor, data) File "/root/anaconda3/envs/torch/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "virtual_gen.py", line 238, in process_one_frame intrinsics=to_batch_tensor(all_cams_intrinsic), transforms=to_batch_tensor(all_cams_from_lidar)) File "/root/anaconda3/envs/torch/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "virtual_gen.py", line 92, in add_virtual_mask indices =mask.reshape(W, H).nonzero() RuntimeError: CUDA error: device-side assert triggered

Could you please give me some suggestions?

these keys have mismatched shape: +-------------------------------+---------------------------------+---------------------------------+ | key | expected shape | loaded shape | +-------------------------------+---------------------------------+---------------------------------+ | backbone.conv_input.0.weight | torch.Size([16, 3, 3, 3, 5]) | torch.Size([3, 3, 3, 5, 16]) | | backbone.conv1.0.conv1.weight | torch.Size([16, 3, 3, 3, 16]) | torch.Size([3, 3, 3, 16, 16]) | | backbone.conv1.0.conv2.weight | torch.Size([16, 3, 3, 3, 16]) | torch.Size([3, 3, 3, 16, 16]) | | backbone.conv1.1.conv1.weight | torch.Size([16, 3, 3, 3, 16]) | torch.Size([3, 3, 3, 16, 16]) | | backbone.conv1.1.conv2.weight | torch.Size([16, 3, 3, 3, 16]) | torch.Size([3, 3, 3, 16, 16]) | | backbone.conv2.0.weight | torch.Size([32, 3, 3, 3, 16]) | torch.Size([3, 3, 3, 16, 32]) | | backbone.conv2.3.conv1.weight | torch.Size([32, 3, 3, 3, 32]) | torch.Size([3, 3, 3, 32, 32]) | | backbone.conv2.3.conv2.weight | torch.Size([32, 3, 3, 3, 32]) | torch.Size([3, 3, 3, 32, 32]) | | backbone.conv2.4.conv1.weight | torch.Size([32, 3, 3, 3, 32]) | torch.Size([3, 3, 3, 32, 32]) | | backbone.conv2.4.conv2.weight | torch.Size([32, 3, 3, 3, 32]) | torch.Size([3, 3, 3, 32, 32]) | | backbone.conv3.0.weight | torch.Size([64, 3, 3, 3, 32]) | torch.Size([3, 3, 3, 32, 64]) | | backbone.conv3.3.conv1.weight | torch.Size([64, 3, 3, 3, 64]) | torch.Size([3, 3, 3, 64, 64]) | | backbone.conv3.3.conv2.weight | torch.Size([64, 3, 3, 3, 64]) | torch.Size([3, 3, 3, 64, 64]) | | backbone.conv3.4.conv1.weight | torch.Size([64, 3, 3, 3, 64]) | torch.Size([3, 3, 3, 64, 64]) | | backbone.conv3.4.conv2.weight | torch.Size([64, 3, 3, 3, 64]) | torch.Size([3, 3, 3, 64, 64]) | | backbone.conv4.0.weight | torch.Size([128, 3, 3, 3, 64]) | torch.Size([3, 3, 3, 64, 128]) | | backbone.conv4.3.conv1.weight | torch.Size([128, 3, 3, 3, 128]) | torch.Size([3, 3, 3, 128, 128]) | | backbone.conv4.3.conv2.weight | torch.Size([128, 3, 3, 3, 128]) | torch.Size([3, 3, 3, 128, 128]) | | backbone.conv4.4.conv1.weight | torch.Size([128, 3, 3, 3, 128]) | torch.Size([3, 3, 3, 128, 128]) | | backbone.conv4.4.conv2.weight | torch.Size([128, 3, 3, 3, 128]) | torch.Size([3, 3, 3, 128, 128]) | | backbone.extra_conv.0.weight | torch.Size([128, 3, 1, 1, 128]) | torch.Size([3, 1, 1, 128, 128]) | +-------------------------------+---------------------------------+---------------------------------+

Hi, I am wondering how can I generate virtual points for nuscenes test set. Since I don't find a .pkl file like "infos_train_10sweeps_withvelo_filter_True.pkl", which seems to be required for generating virtual points with command

python virtual_gen.py --info_path data/nuScenes/infos_train_10sweeps_withvelo_filter_True.pkl  MODEL.WEIGHTS centernet2_checkpoint.pth 

Thanks for your work! But it takes me about 3 days to train MVP (with 4 V100). I changed your env to spconv 2 and CUDA11.1. If using CUDA10.2 and spconv 1 mentioned in your install guide, it takes longer. So could you please tell me how long do you take? I have enough GPU memories and is it possible to accelerate the training? Thanks!

Please see https://github.com/tianweiy/CenterPoint/blob/2145ada1f96b7b4b8f19f3b0012aab589962806f/det3d/core/sampler/preprocess.py#L838

def global_scaling_v2(gt_boxes, points, min_scale=0.95, max_scale=1.05):
    noise_scale = np.random.uniform(min_scale, max_scale)
    points[:, :3] *= noise_scale
    gt_boxes[:, :-1] *= noise_scale
    return gt_boxes, points

"gt_boxes" is [box_number,9]. 9 corresponds to x, y, z, w, h, l, vx, vy, r. So why is the vx and vy of gt_boxes scaled? Is this a bug? And by the way, where can I find the data format description of ground truth annotations (the definition of each dimension)? I can't find it in NuScenes' website and github. Thanks!

Thanks for your great work and released code! I am wondering if there are any methods to align the camera images in nuScences with those in nuImages. Since I observed that roughly 76% virtual points, selected according to CenterNet2's instance segmentation masks, are discarded because that they don't correspond to any instances annotated in the nuScenes. Hence, I am wondering the influence of the discrepancy between the instances detected by CenterNet2 and those annotated in the nuScenes, and would like to know if there are any possible ways to directly obtain the 2D annotations of camera images within the nuScenes?

I have two questions about the abation study in your paper:

1. What is split voxelization?

2. Does your repo contain the code about split voxelization or two-stage method?

Hi, I just found there are about 2000 sweeps of virtual points missing.

Here are the number of files in sweeps/LIDAR_TOP/

ls sweeps/LIDAR_TOP/ | wc -l
297737

Here are the number of files in sweeps/LIDAR_TOP_VIRTUAL/

ls data/nuscenes/sweeps/LIDAR_TOP_VIRTUAL/ | wc -l
295489

I directly download and extract the data from the official link here. I am pretty sure I didn't mess with the data. I wonder is this normal?

Thanks!

Hi! Thank you for your great work!

I am following the instructions in getting_started. However, I fail to understand the difference between creating data with virtual points (MVP) and creating data without virtual points (CenterPoint Baseline). Based on the instructions, the command for MVP is

# nuScenes
python tools/create_data.py nuscenes_data_prep --root_path=NUSCENES_TRAINVAL_DATASET_ROOT --version="v1.0-trainval" --nsweeps=10 --virtual True 

And the command for CenterPoint is

# nuScenes
python tools/create_data.py nuscenes_data_prep --root_path=NUSCENES_TRAINVAL_DATASET_ROOT --version="v1.0-trainval" --nsweeps=10 --virtual False 

It seems to me that the only difference is --virtual, which is only used in creating gt sampling here. But according to the instructions, the improvements of gt sampling on nuscenes is marginal and thus can be simply discarded. If that is the case, it seems to me there is no real difference between creating data with or without virtual points.

Am I missing something here? Thanks!

Hi! Thank you for your great work! I'm trying to reproduce MVP(CenterPoint-VoxelNet) results with OpenPCDet, but the results seems not good enough compared to your official results (mAP: 66, NDS:69.9). I modify points loading related functions and points dimensions (5 to 22) related code just like yours. I check the voxel features and nothing seems wrong to me. Could it be test time augmentation that leads to this inferior results? It seems that there's no TTA in default config file of CenterPoint in OpenPCDet. Did you get the result of mAP 66 with TTA?

Hi, I notice that you generate instance masks for KITTI using the maskformer pretrained on coco-panoptic. I am wondering whether the domain gap between COCO and KITTI will lead to unsatisfied instance segmentation performances. Because I notice that the improvements on KITTI is not as drastic as on nuScenes.