Awesome-LiDAR-Camera-Calibration

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

Introduction
Target-based methods
Targetless methods
- Motion-based methods
- Scene-based-methods
  - Traditional Methods
  - Deep-learning Methods
Other Toolboxes

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

1. Target-based methods

Paper	Target	Feature	Optimization	Toolbox	Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004	checkerboard	C:Plane (a), L: pts in plane (m)	point-to-plane	CamLaserCalibraTool	CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005	checkerboard	C: Plane (a), L: Plane (m)	plane(n/d) correspondence, point-to-plane	LCCT	*
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010	checkerboard	C: plane (a), L: pts in plane (m)	point-to-plane	cam_lidar_calib	*
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017	cardboard + ArUco	C: 3D corners (a), L: 3D corners (m)	ICP	lidar_camera_calibration	*
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017	checkerboard	C: 2D corners (a), L: 3D corners (a)	PnP, angle difference	ILCC	*
Extrinsic Calibration of Lidar and Camera with Polygon, 2018	regular cardboard	C: 2D edge, corners (a), L: 3D edge, pts in plane (a)	point-to-line, point-inside-polygon	ram-lab/plycal	*
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018	checkerboard	C: 3D edge, plane(a), L: 3D edge, pts in plane (a)	direcion/normal, point-to-line, point-to-plane	Matlab LiDAR Toolbox	*
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020	cardboard with ArUco	C: 2d corners (a), L: 3D corners (a)	PnP, IOU	github	*
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020	checkerboard	C: 2D corners (a), L: 3D corners (a)	PnP	ACSC	*
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021	cardboard with circle & Aruco	C: 3D points (a), L: 3D points (a)	ICP	velo2cam_ calibration	*

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper	Feature	Optimization	Toolbox	Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018	C: motion (ICP), L: motion (VO)	hand-eye calibration	*	*

2.2. Scene-based methods

2.2.1. Traditional methods

Paper	Feature	Optimization	Toolbox	Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012	C：grayscale, L: reflectivity	mutual information, BB steepest gradient ascent	Extrinsic Calib	*
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013	C:grayscale, L: reflectivity, noraml	normalized MI, particle swarm	*	*
Automatic Online Calibration of Cameras and Lasers, 2013	C: Canny edge, L: depth-discontinuous edge	correlation, grid search	*	*
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020	semantic centroid	PnP	*	*
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021	C: edge(grayscale), L: edge (reflectivity, depth projection)	ICP, coordinate descent algorithms	CamVox	*
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021	C:Canny edge(grayscale), L: depth-continuous edge	point-to-line, Gaussian-Newton	livox_camera_calib	*
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021	C:straight line, L: straight line	perspective3-lines (P3L)	*	CN

2.2.2. Deep-learning methods

Pape	Toolbox	Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV	regnet	*
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS	CalibNet	*

3. Other toolboxes

Toolbox	Introduction	Note
Apollo sensor calibration tools	targetless method, no source code	CN
Autoware camera lidar calibrator	pick points mannually, PnP	*
Autoware calibration camera lidar	checkerboard, similar to LCCT	CN
livox_camera_lidar_calibration	pick points mannually, PnP	*

A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

Website, Tutorials, and Docs Uncertainty Toolbox A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualizatio

1.4k Dec 28, 2022

CSAC - Collaborative Semantic Aggregation and Calibration for Separated Domain Generalization

CSAC Introduction This repository contains the implementation code for paper: Co

5 Jul 22, 2022

Aiming at the common training datsets split, spectrum preprocessing, wavelength select and calibration models algorithm involved in the spectral analysis process

Aiming at the common training datsets split, spectrum preprocessing, wavelength select and calibration models algorithm involved in the spectral analysis process, a complete algorithm library is established, which is named opensa (openspectrum analysis).

50 Jan 7, 2023

The comma.ai Calibration Challenge!

Welcome to the comma.ai Calibration Challenge! Your goal is to predict the direction of travel (in camera frame) from provided dashcam video. This rep

697 Jan 5, 2023

The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Representative Batch Normalization (RBN) with Feature Calibration The official implementation of the CVPR2021 oral paper: Representative Batch Normali

76 Nov 9, 2022

Improving Calibration for Long-Tailed Recognition (CVPR2021)

MiSLAS Improving Calibration for Long-Tailed Recognition Authors: Zhisheng Zhong, Jiequan Cui, Shu Liu, Jiaya Jia [arXiv] [slide] [BibTeX] Introductio

116 Dec 20, 2022

19 Apr 28, 2021

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

SNN_Calibration Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021 Feature Comparison of SNN calibration: Features SNN Direct Tr

60 Dec 27, 2022

the code for paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration"

EOW-Softmax This code is for the paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration". Accepted by ICCV21. Usage Commnd exa

36 Dec 2, 2022

TierIV calibration tools

(Disclaimer: I am one of the developers)

In TierIV we are developing calibration tools for autonomous driving and as part of those we developed a target-based camera-lidar method and a UI-based manual calibration tool. Although they do not introduce much novelty, they provide quite good results and as we keep developing new calibration tools for different use-cases, we plan to support the repository for a long period of time.

I came to see this curated list of methods, tools, and papers while doing my literature review so I would love it if our software could be included here !


Best regards,

Kenzo

opened by knzo25 1

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Related tags

Overview

Awesome-LiDAR-Camera-Calibration

0. Introduction

1. Target-based methods

2. Targetless methods

2.1. Motion-based methods

2.2. Scene-based methods

2.2.1. Traditional methods

2.2.2. Deep-learning methods

3. Other toolboxes

You might also like...

A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

CSAC - Collaborative Semantic Aggregation and Calibration for Separated Domain Generalization

Aiming at the common training datsets split, spectrum preprocessing, wavelength select and calibration models algorithm involved in the spectral analysis process

The comma.ai Calibration Challenge!

The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Improving Calibration for Long-Tailed Recognition (CVPR2021)

Improving Calibration for Long-Tailed Recognition (CVPR2021)

Pytorch Implementation of Spiking Neural Networks Calibration, ICML 2021

the code for paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration"

Comments

TierIV calibration tools

Owner

Camera calibration & 3D pose estimation tools for AcinoSet

CTRL-C: Camera calibration TRansformer with Line-Classification

Omnidirectional camera calibration in python

This is the official implementation of 3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection, built on SECOND.

CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection

arxiv-sanity, but very lite, simply providing the core value proposition of the ability to tag arxiv papers of interest and have the program recommend similar papers.

The tl;dr on a few notable transformer/language model papers + other papers (alignment, memorization, etc).

Camera-caps - Examine the camera capabilities for V4l2 cameras

Collection of TensorFlow2 implementations of Generative Adversarial Network varieties presented in research papers.

S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration (CVPR 2021)