How do I conduct the experiments in Table 1 mentioned in the paper?

I change the below code but The result is so bad.

How can I get the Table 1 result?

Code:

def evaluate(label_path,
             result_path,
             label_split_file,
             current_class=0,
             coco=False,
             score_thresh=-1):
    dt_annos = kitti.get_label_annos(result_path)
    if score_thresh > 0:
        dt_annos = kitti.filter_annos_low_score(dt_annos, score_thresh)
    val_image_ids = _read_imageset_file(label_split_file)
    gt_annos = kitti.get_label_annos(label_path, val_image_ids)
    dt_annos = kitti.get_label_annos(label_path, val_image_ids)
    if coco:
        return get_coco_eval_result(gt_annos, dt_annos, current_class)
    else:
        return get_official_eval_result(gt_annos, dt_annos, current_class)

Result: 2021-08-13 18:43:24,476 INFO Car [email protected], 0.70, 0.70: bbox AP:100.0000, 100.0000, 100.0000 bev AP:0.0865, 0.1172, 0.2115 3d AP:0.0865, 0.1172, 0.2115 aos AP:100.00, 100.00, 100.00 Car [email protected], 0.70, 0.70: bbox AP:100.0000, 100.0000, 100.0000 bev AP:0.0238, 0.0322, 0.0582 3d AP:0.0238, 0.0322, 0.0582 aos AP:100.00, 100.00, 100.00 Car [email protected], 0.50, 0.50: bbox AP:100.0000, 100.0000, 100.0000 bev AP:0.0865, 0.1172, 0.2115 3d AP:0.0865, 0.1172, 0.2115 aos AP:100.00, 100.00, 100.00 Car [email protected], 0.50, 0.50: bbox AP:100.0000, 100.0000, 100.0000 bev AP:0.0238, 0.0322, 0.0582 3d AP:0.0238, 0.0322, 0.0582 aos AP:100.00, 100.00, 100.00

Hi, I have reproduced your result on KITTI. I can't find your submission on the nuScenes leaderboard, so I wonder if you have done experiments on the nuScenes dataset. If yes, could you share your accuracy? Will you release the code for nuScenes?

Thanks for your excellent work~

Hi, @xinzhuma Thanks for your amazing work! I trained your code and got the following results: Do you have any suggestions about unstable training results? I think you have made code reproducible in set_random_seed.

I train the model on single GPU and never use the distributed training. The batch size is set at 8. The detection results are as follows: Car [email protected], 0.70, 0.70: bbox AP:89.6069, 86.3804, 78.2012 bev AP:23.5712, 18.8870, 15.9993 3d AP:16.8725, 13.4053, 12.3370 aos AP:88.85, 84.82, 76.00 Car [email protected], 0.70, 0.70: bbox AP:94.9005, 89.4963, 80.5794 bev AP:21.5961, 16.3269, 13.8877 3d AP:14.9669, 11.0160, 9.5246 aos AP:93.97, 87.69, 78.04

Hi, @xinzhuma Many thanks for the great work! When I do the evaluation, it only tests on the Car. When I add the Pedestrian and Cyclist, the results are 0. So does the model only train on Car?

Hi, @xinzhuma Q1: It seems that code can't run on 30x0 GPU. I guess your code can't support cuda11 now.

Q2:It can't set random seed to do further experiment. Since you have set fixed random seed now, but it is not really work as expected. Each time you train one model, the final results are different. Do you have any suggestions about it? thanks very much! (This is strange, because we always use this method to make model reproducted)

图片预处理代码中当flip时，并未对x取反，这样会导致box3d在图中投影不对。这是一个bug吗？ https://github.com/xinzhuma/monodle/blob/273d801a11ca048283abce6cc6a9898e76322f8e/lib/datasets/kitti/kitti_dataset.py#L165

是否要加入：

object.pos[0] = -object.pos[0]

@xinzhuma

您好，感谢您的优秀工作。我在跑代码的过程中遇到了以下几个问题。

1. 测试预训练模型时，readme中的预训练模型是多卡训练的，单卡测试会报错。解决办法：在monodle/lib/helpers/save_helper.py:32 加入：

    model = torch.nn.DataParallel(model).cuda()  # 强制让模型多卡

2. 训练时，self.stats['train'] = {}这里会报错。解决办法：在amonodle/lib/helpers/trainer_helper.py:90前面加入self.stats={}。
3. 2080ti上，2块卡训练，batch_size=16，显卡利用率大部分时间是0%，训练很慢。在代码中没看到您用torch的Dataloader，不清楚怎么加num_workers。请问有遇到类似的问题吗？谢谢

Hi, @xinzhuma Have you test backbone such as DLA102? When change config file dla34 to dla102, the 3D AP is only 10 which is 6% performance degradation compared to default config setting. Wishing for your reply.

We propose a method based on the monodle and need to do ablation study to evaluate the effect of our design. We obtain the different model when using the same code to train the network , so the detection results are vary largely. In this case, how to do the ablation study? Do you have a solution? We train the model on a single GPU.

您好，感谢您的优秀工作。关于Training Samples这块有一些疑问，我看代码中是直接按照距离进行了过滤，并没有进行ignore处理 https://github.com/xinzhuma/monodle/blob/main/lib/datasets/kitti/kitti_dataset.py#L201

直接过滤不进行ignore处理，远距离样本不就当成负样本，这样不会有问题吗？

Hello, when reading the tag data, if I change the filter conditions of the instance level, does it mean that the distance of the model I trained focus is different, and will the level filtering hyperparameter affect the evaluation process?

Hello!

Thank you for your work. I have a question to ask you. I made my own dataset in KITTI format, but the image size is 1280 * 720. How do I need to modify the corresponding code

Why is the accuracy of the moderate car reported in the paper is 12.28, but in this repository reports 13.66 and 13.72？Are the tables in this repository describing the results on the val set?

why when I train monodle： (monodle) G:\WWProject\monodle\experiments\example>python ../../tools/train_val.py --config kitti_example.yaml 2022-10-11 10:46:15,326 INFO ################### Training ################## 2022-10-11 10:46:15,326 INFO Batch Size: 16 2022-10-11 10:46:15,326 INFO Learning Rate: 0.001250 Then it took more than half an hour to appear in the training epoch progress bar, I changed the original monodle to a single GPU run, mine is a single 3090 epochs: 0%| | 0/140 [00:00<?, ?it/s] iters: 0%| | 0/232 [00:00<?, ?it/s]

Hi,thanks for your great work. when we use dim_aware_loss to train dim we found that there is some bug in dim_aware_loss.

https://github.com/xinzhuma/monodle/blob/e426aa65fdc7ceedcaab0d637acf3d3425d0736c/lib/losses/dim_aware_loss.py#L8

loss /= dimension may change the gradient direction of one of the parameters (h, w, l). For example, when h < 0, the gradient is of |h - h*| is negative, so that the graident changes to positive when divided by h and this will cause the loss to increase when using gradient descent to update parameters. The gradient direction of all parameters should not be changed after dividing by dimension. We recommand that this line be changed to loss/=torch.abs(dimension).