News: We released the codebase v0.12.0.
In the recent nuScenes 3D detection challenge of the 5th AI Driving Olympics in NeurIPS 2020, we obtained the best PKL award and the second runner-up by multi-modality entry, and the best vision-only results. Code and models will be released soon!
Documentation: https://mmdetection3d.readthedocs.io/
Introduction
English | 简体中文
The master branch works with PyTorch 1.3+.
MMDetection3D is an open source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D detection. It is a part of the OpenMMLab project developed by MMLab.
Major features
-
Support multi-modality/single-modality detectors out of box
It directly supports multi-modality/single-modality detectors including MVXNet, VoteNet, PointPillars, etc.
-
Support indoor/outdoor 3D detection out of box
It directly supports popular indoor and outdoor 3D detection datasets, including ScanNet, SUNRGB-D, Waymo, nuScenes, Lyft, and KITTI. For nuScenes dataset, we also support nuImages dataset.
-
Natural integration with 2D detection
All the about 300+ models, methods of 40+ papers, and modules supported in MMDetection can be trained or used in this codebase.
-
High efficiency
It trains faster than other codebases. The main results are as below. Details can be found in benchmark.md. We compare the number of samples trained per second (the higher, the better). The models that are not supported by other codebases are marked by
×.Methods MMDetection3D OpenPCDet votenet Det3D VoteNet 358 × 77 × PointPillars-car 141 × × 140 PointPillars-3class 107 44 × × SECOND 40 30 × × Part-A2 17 14 × ×
Like MMDetection and MMCV, MMDetection3D can also be used as a library to support different projects on top of it.
License
This project is released under the Apache 2.0 license.
Changelog
v0.12.0 was released in 1/4/2021. Please refer to changelog.md for details and release history.
Benchmark and model zoo
Supported methods and backbones are shown in the below table. Results and models are available in the model zoo.
Support backbones:
- PointNet (CVPR'2017)
- PointNet++ (NeurIPS'2017)
- RegNet (CVPR'2020)
Support methods
- SECOND (Sensor'2018)
- PointPillars (CVPR'2019)
- FreeAnchor (NeurIPS'2019)
- VoteNet (ICCV'2019)
- H3DNet (ECCV'2020)
- 3DSSD (CVPR'2020)
- Part-A2 (TPAMI'2020)
- MVXNet (ICRA'2019)
- CenterPoint (CVPR'2021)
- SSN (ECCV'2020)
- ImVoteNet (CVPR'2020)
| ResNet | ResNeXt | SENet | PointNet++ | HRNet | RegNetX | Res2Net | |
|---|---|---|---|---|---|---|---|
| SECOND | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| PointPillars | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| FreeAnchor | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| VoteNet | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| H3DNet | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| 3DSSD | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| Part-A2 | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| MVXNet | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| CenterPoint | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| SSN | ☐ | ☐ | ☐ | ✗ | ☐ | ✓ | ☐ |
| ImVoteNet | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
Other features
Note: All the about 300+ models, methods of 40+ papers in 2D detection supported by MMDetection can be trained or used in this codebase.
Installation
Please refer to getting_started.md for installation.
Get Started
Please see getting_started.md for the basic usage of MMDetection3D. We provide guidance for quick run with existing dataset and with customized dataset for beginners. There are also tutorials for learning configuration systems, adding new dataset, designing data pipeline, customizing models, customizing runtime settings and Waymo dataset.
Citation
If you find this project useful in your research, please consider cite:
@misc{mmdet3d2020,
title={{MMDetection3D: OpenMMLab} next-generation platform for general {3D} object detection},
author={MMDetection3D Contributors},
howpublished = {\url{https://github.com/open-mmlab/mmdetection3d}},
year={2020}
}
Contributing
We appreciate all contributions to improve MMDetection3D. Please refer to CONTRIBUTING.md for the contributing guideline.
Acknowledgement
MMDetection3D is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new 3D detectors.
Projects in OpenMMLab
- MMCV: OpenMMLab foundational library for computer vision.
- MMClassification: OpenMMLab image classification toolbox and benchmark.
- MMDetection: OpenMMLab detection toolbox and benchmark.
- MMDetection3D: OpenMMLab next-generation platform for general 3D object detection.
- MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
- MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
- MMTracking: OpenMMLab video perception toolbox and benchmark.
- MMPose: OpenMMLab pose estimation toolbox and benchmark.
- MMEditing: OpenMMLab image and video editing toolbox.
PGD：
As you can see. They all detect too many 3D boxes. PGD even has a bunch of 3D boxes that shouldn't be there. What causes this problem? How to solve it?
2、I wrote test instructions to test according to the official documents and git instructions. Unfortunately, it can only be viewed and saved one by one. Batch tests cannot be completed, saved, and then viewed. Is there any solution to this problem? Maybe I'm not doing it right? I enclose mine here：
python tools/test.py configs/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_kitti-mono3d.py /media/lee/DATA/project/FCOS3D/latest.pth --show --show-dir /media/lee/DATA/project/dataset/FCOS
python tools/test.py configs/pgd/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d.py checkpoints/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d_20211022_102608-8a97533b.pth --show --show-dir /media/lee/DATA/project/dataset/PGD
I look forward to your early reply! Thank you very much.
![[Feature] Support PointRCNN RPN and RCNN module](https://avatars.githubusercontent.com/u/79644370?v=4)
![[Fix] fix a bug that may cause compilation failure of dynamic voxelization when using GPUs with compute capability lower than 6.x](https://avatars.githubusercontent.com/u/37722266?v=4)
![[Feature] groupfree3d objectness loss](https://avatars.githubusercontent.com/u/27985664?v=4)
![[Bug] Result is saved to /tmp/tmpmp_yh0x5/results.pkl.](https://avatars.githubusercontent.com/u/88964298?v=4)
![[Fix] Fix `cam_list` for `WaymoDataset`](https://avatars.githubusercontent.com/u/55445986?v=4)
![[CodeCamp #1503] Use mmeval.InstanceSeg](https://avatars.githubusercontent.com/u/31173671?v=4)
![[Fix] update SECOND checkpoints link and README](https://avatars.githubusercontent.com/u/36950400?v=4)
539 Jan 7, 2023
24 Dec 28, 2022
405 Jan 4, 2023
688 Jan 6, 2023
147 Dec 3, 2022
185 Dec 26, 2022
3 Nov 11, 2022
2.2k Jan 5, 2023
743 Jan 1, 2023
6 Nov 4, 2022
6.6k Jan 3, 2023
5 Dec 10, 2022
3 Jan 26, 2022
5 Aug 25, 2020
41 Nov 23, 2022
99 Dec 11, 2022
443 Dec 6, 2022