RGGNet: Tolerance Aware LiDAR-Camera Online Calibration With Geometric Deep Learning and Generative Model

Yuan, Kaiwen; Guo, Zhenyu; Wang, Z. Jane

doi:10.1109/lra.2020.3026958

Cited by 75 publications

(48 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above deep learning calibration methods ignore the tolerance within the error bounds. RGGNet [14] utilized the Riemannian geometry and deep generative model to build a tolerance-aware loss function.…”

Section: Deep Learning Approachmentioning

confidence: 99%

“…The initial calibration off-range ∆T was (±1.5 m, ±20 • ). To compare our method with other learning-based (CNN-based) methods, we utilized the same four test datasets [14] on the raw recordings of the KITTI dataset. Each test dataset was independent of the training dataset with the following test name configurations:…”

Section: Dataset Preparationmentioning

confidence: 99%

“…Besides, we applied another two se(3) error based evaluation metrics mentioned in [14]: mean error (MSEE) se(3) and mean re-calibration rate (MRR). MRR reflects how much noise us compensated by the re-calibration.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…Thus, some efforts attempted to adopt deep learning to learn the 6-DoF rigid transformation between LiDAR and camera data. Deep learning-based extrinsic calibration methods [12][13][14][15] also demonstrate flexibility and adaptability in changing environments. Some methods [12,13] cannot meet the performance requirements of real-world applications.…”

Section: Introductionmentioning

confidence: 99%

“…Some methods [12,13] cannot meet the performance requirements of real-world applications. Some methods [14,15] regress the parameters directly, which introduces training difficulties. The gap between the performances of deep learning-based methods and the practical demands is being bridged.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

Wang

Dou

et al. 2021

Sensors

View full text Add to dashboard Cite

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization workflow, which is time-consuming and requires prior information. Convolutional neural networks (CNNs) can regress the six degrees of freedom (6-DOF) extrinsic parameters from raw LiDAR and image data. However, these CNN-based methods just learn the representations of the projected LiDAR and image and ignore the correspondences at different locations. The performances of these CNN-based methods are unsatisfactory and worse than those of non-CNN methods. In this paper, we propose a novel CNN-based LiDAR-camera extrinsic calibration algorithm named CFNet. We first decided that a correlation layer should be used to provide matching capabilities explicitly. Then, we innovatively defined calibration flow to illustrate the deviation of the initial projection from the ground truth. Instead of directly predicting the extrinsic parameters, we utilize CFNet to predict the calibration flow. The efficient Perspective-n-Point (EPnP) algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D–3D correspondences constructed by the calibration flow. Due to its consideration of the geometric information, our proposed method performed better than the state-of-the-art CNN-based methods on the KITTI datasets. Furthermore, we also tested the flexibility of our approach on the KITTI360 datasets.

show abstract

Section: Deep Learning Approachmentioning

confidence: 99%

Section: Dataset Preparationmentioning

confidence: 99%

Section: Evaluation Metricsmentioning

confidence: 99%