Please cite the following paper if this repo helps your research:
@InProceedings{Luo_2021_ICCV,
author = {Luo, Chenxu and Yang, Xiaodong and Yuille, Alan},
title = {Exploring Simple 3D Multi-Object Tracking for Autonomous Driving},
booktitle = {International Conference on Computer Vision (ICCV)},
year = {2021}
}
License
Copyright (C) 2021 QCraft. All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International). The code is released for academic research use only. For commercial use, please contact [email protected].
when i train the model using command line in the tutorial, I got this :
Traceback (most recent call last): File "./tools/train.py", line 13, in <module> from det3d.models import build_detector File "/home/lz/task3/simtrack/simtrack/det3d/models/__init__.py", line 13, in <module> from .detectors import * # noqa: F401,F403 File "/home/lz/task3/simtrack/simtrack/det3d/models/detectors/__init__.py", line 3, in <module> from .point_pillars_tracking import PointPillarsTracking ModuleNotFoundError: No module named 'det3d.models.detectors.point_pillars_tracking'
The error occurs because there is no point_pillars_tracking.py in /home/lz/task3/simtrack/simtrack/det3d/models/detectors, but a voxelnet.py which has been commented out.
Can anybodyhelp me ?
I have one question, does the model structure of the simTrack is completely the same as CenterPoint?
And the only difference is the post-processing. Am I right? Do I miss something important?
We use A100 for training, 4 gpu cards, 4 epochs , the rest of the parameters have not changed, the training 200 epochs still does not achieve the effect. What training skills do you have? such as the learning rate modify and so on
We train the model that comes with simtrack. After training for 20 epochs, we continue to train based on the model, and the loss will continue to decline. why did you stop training?
I sincerely look forward to your reply.
File "/mnt/data02/wzy/simtrack/det3d/models/detectors/init.py", line 3, in
from .point_pillars_tracking import PointPillarsTracking
ModuleNotFoundError: No module named 'det3d.models.detectors.point_pillars_tracking'
i have looked through the det3d ,and don`t find it. What should i do for it ?
Thanks your fantastic work first, and I have a problem while view the code, did you do anything about motion compensation or undistort the pointcloud? I saw you use the matrix between two sweep lidar points to concat the points, but I did not see motion process when concat two sweeps lidar points, maybe I missed. If do not do this, is there any distort problem for the objects with big velocity?
I have a gpu card with only 12GB video memory. The process failed with out-of-memory error when I tried to run with default config. What should I change in the default config to run your model normally?
Nice work. I noticed that your implemented DynamicPillarFeatureNet requests the raw points as an input and not the output of the voxelizer. I wonder how to train using this pipeline?
I don't understand the actual physical meaning of “tracking_batch_hm”.I also don’t understand why we need to execute this way instead of directly using batch_hm or prev_hm?
I have another question that if the displacement of an object is relatively large, then the position of this object in the previous centerness map will be farther away from that in the current centerness map.(In other words, there is no intersection between the representation of this object in the previous centerness map and the current centerness map.)
So after NMS operation, will this object be considered as a new object?
**Dear authors:
I really appreciate your great work, and i'm tring to use it in some of my projects. I train your model on 4 3090 and this model is trained for 187 epochs on nuscenes, but the tracking result is bad, just like this picture, What is the cause?
hello, when I tried to train a model, I just got this error, the traceback is following:
2022-04-24 18:16:11,886 - INFO - Distributed training: False
2022-04-24 18:16:11,886 - INFO - torch.backends.cudnn.benchmark: False
2022-04-24 18:16:11,900 - INFO - Backup source files to SAVE_DIR/det3d
Traceback (most recent call last):
File "./tools/train.py", line 146, in
main()
File "./tools/train.py", line 119, in main
model = build_detector(cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)
File "/simtrack/det3d/models/builder.py", line 53, in build_detector
return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/simtrack/det3d/models/builder.py", line 21, in build
return build_from_cfg(cfg, registry, default_args)
File "/simtrack/det3d/utils/registry.py", line 66, in build_from_cfg
"{} is not in the {} registry".format(obj_type, registry.name)
KeyError: 'PointPillars is not in the detector registry'
Traceback (most recent call last):
File "/anaconda3/envs/simtrack/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/anaconda3/envs/simtrack/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/anaconda3/envs/simtrack/lib/python3.6/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/anaconda3/envs/simtrack/lib/python3.6/site-packages/torch/distributed/launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '[/anaconda3/envs/simtrack/bin/python', '-u', './tools/train.py', '--local_rank=0', 'examples/point_pillars/configs/nusc_all_pp_centernet_tracking.py', '--work_dir', 'SAVE_DIR']' returned non-zero exit status 1.
the order is: python -m torch.distributed.launch --nproc_per_node=1 ./tools/train.py examples/point_pillars/configs/nusc_all_pp_centernet_tracking.py --work_dir SAVE_DIR
I used the default model to eval, but triger a warming and a AssertionError, sincerely look forward to your answer. Also, when will you release the full version of the code?
Reverse indexing ...
Done reverse indexing in 10.4 seconds.
/data/simtrack/det3d/core/bbox/geometry.py:160: NumbaWarning:
Compilation is falling back to object mode WITH looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: No implementation of function Function() found for signature:
Of which 20 did not match due to:
Overload of function 'getitem': File: : Line N/A.
With argument(s): '(array(float64, 3d, C), Tuple(slice<a:b>, list(int64)<iv=None>, slice<a:b>))':
No match.
Of which 2 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 166.
With argument(s): '(array(float64, 3d, C), Tuple(slice<a:b>, list(int64)<iv=None>, slice<a:b>))':
Rejected as the implementation raised a specific error:
NumbaTypeError: unsupported array index type list(int64)<iv=None> in Tuple(slice<a:b>, list(int64)<iv=None>, slice<a:b>)
raised from /home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/numba/core/typing/arraydecl.py:73
During: typing of intrinsic-call at /data/simtrack/det3d/core/bbox/geometry.py (179)
File "det3d/core/bbox/geometry.py", line 179:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
@numba.jit
/data/simtrack/det3d/core/bbox/geometry.py:160: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "points_in_convex_polygon_jit" failed type inference due to: Cannot determine Numba type of <class 'numba.core.dispatcher.LiftedLoop'>
File "det3d/core/bbox/geometry.py", line 196: def points_in_convex_polygon_jit(points, polygon, clockwise=True):
For more information visit https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit
File "det3d/core/bbox/geometry.py", line 171:
def points_in_convex_polygon_jit(points, polygon, clockwise=True):
# first convert polygon to directed lines
num_points_of_polygon = polygon.shape[1]
^
Reverse indexing ...
Done reverse indexing in 8.5 seconds. ======
Initializing nuScenes tracking evaluation
Loaded results from /data/simtrack_output/tracking_results.json. Found detections for 6019 samples.
Loading annotations for val split from nuScenes version: v1.0-trainval
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6019/6019 [00:06<00:00, 871.16it/s]
Loaded ground truth annotations for 6019 samples.
Filtering tracks
=> Original number of boxes: 227984
=> After distance based filtering: 190099
=> After LIDAR points based filtering: 190099
=> After bike rack filtering: 189972
Filtering ground truth tracks
=> Original number of boxes: 142261
=> After distance based filtering: 103564
=> After LIDAR points based filtering: 93885
=> After bike rack filtering: 93875
Accumulating metric data...
Computing metrics for class bicycle...
Traceback (most recent call last):
File "./tools/val_nusc_tracking.py", line 202, in
tracking()
File "./tools/val_nusc_tracking.py", line 148, in tracking
dataset.evaluation_tracking(copy.deepcopy(predictions), output_dir=args.work_dir, testset=False)
File "/data/simtrack/det3d/datasets/nuscenes/nuscenes.py", line 382, in evaluation_tracking
metrics_summary = nusc_eval.main()
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 205, in main
metrics, metric_data_list = self.evaluate()
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 135, in evaluate
accumulate_class(class_name)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 131, in accumulate_class
curr_md = curr_ev.accumulate()
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/nuscenes/eval/tracking/algo.py", line 156, in accumulate
assert unachieved_thresholds + duplicate_thresholds + len(thresh_metrics) == self.num_thresholds
AssertionError
2022-03-17 14:21:36,906 - INFO - Start running, host: yangjinrong@tracking-q5x64-32246-worker-0, work_dir: /data/simtrack_output
2022-03-17 14:21:36,907 - INFO - workflow: [('train', 1), ('val', 1)], max: 20 epochs
Traceback (most recent call last):
File "./tools/train.py", line 141, in
main()
File "./tools/train.py", line 136, in main
logger=logger,
File "/data/simtrack/det3d/torchie/apis/train.py", line 206, in train_detector
trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank)
File "/data/simtrack/det3d/torchie/trainer/trainer.py", line 527, in run
epoch_runner(data_loaders[i], self.epoch, **kwargs)
File "/data/simtrack/det3d/torchie/trainer/trainer.py", line 393, in train
self.model, data_batch, train_mode=True, **kwargs
File "/data/simtrack/det3d/torchie/trainer/trainer.py", line 356, in batch_processor
losses = model(example, return_loss=True)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 511, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/simtrack/det3d/models/detectors/point_pillars.py", line 48, in forward
x = self.extract_feat(data)
File "/data/simtrack/det3d/models/detectors/point_pillars.py", line 29, in extract_feat
x = self.neck(x)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/simtrack/det3d/models/necks/rpn.py", line 142, in forward
ups.append(self.deblocksi - self._upsample_start_idx)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/simtrack/det3d/models/utils/misc.py", line 82, in forward
input = module(input)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/modules/activation.py", line 102, in forward
return F.relu(input, inplace=self.inplace)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/nn/functional.py", line 1119, in relu
result = torch.relu(input)
RuntimeError: CUDA out of memory. Tried to allocate 64.00 MiB (GPU 2; 10.76 GiB total capacity; 9.76 GiB already allocated; 47.44 MiB free; 9.88 GiB reserved in total by PyTorch)
^CProcess Process-10:
^CProcess Process-9:
Process Process-9:
Process Process-9:
Process Process-9:
Process Process-3:
Process Process-2:
Process Process-5:
Traceback (most recent call last):
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/subprocess.py", line 1019, in wait
Process Process-2:
Process Process-10:
Process Process-5:
Process Process-10:
Process Process-2:
Process Process-10:
Process Process-5:
Process Process-1:
Process Process-7:
Process Process-5:
Process Process-9:
Process Process-5:
Process Process-4:
Process Process-8:
Process Process-8:
Process Process-8:
Process Process-4:
Process Process-4:
Process Process-1:
Process Process-3:
Process Process-1:
Process Process-6:
Process Process-1:
Process Process-7:
Process Process-7:
return self._wait(timeout=timeout)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/subprocess.py", line 1653, in _wait
(pid, sts) = self._try_wait(0)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/subprocess.py", line 1611, in _try_wait
(pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/distributed/launch.py", line 261, in
main()
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/site-packages/torch/distributed/launch.py", line 254, in main
process.wait()
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/subprocess.py", line 1032, in wait
self._wait(timeout=sigint_timeout)
File "/home/yangjinrong/miniconda3/envs/det3d/lib/python3.7/subprocess.py", line 1647, in _wait
time.sleep(delay)
KeyboardInterruptq
Dear authors:
I really appreciate your great work, and i'm tring to use it in some of my projects. I am wondering how to visualize your result just like the first gif of your README.
Thanks for your answer!
This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.