End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles

Rachman, Arya; Seiler, Jurgen; Kaup, André

doi:10.1109/iv55152.2023.10186613

Cited by 2 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In these algorithms, the ResNet of different depths will be used to meet network design requirements. CalibNet [44], LCCNet [46], CALNet [39], CFNet [59], Calib-RCNN [57], CalibDNN [43], etc., all adopt ResNet-18, while RGGNet [45], CaLiCaNet [83] etc., use ResNet-50 to obtain more robust features.…”

Section: Bf Depth Disparitymentioning

confidence: 99%

“…Similar to the design in RegNet, the RGB branch uses pre-trained ResNet-18, while in the depth branch, the feature channels of ResNet-18 are halved for new training.In these algorithms, the ResNet of different depths will be used to meet network design requirements. CalibNet[44], LCCNet[46], CALNet[39], CFNet[59], Calib-RCNN[57], CalibDNN[43], etc., all adopt ResNet-18, while RGGNet[45], CaLiCaNet[83] etc., use ResNet-50 to obtain more robust features.Attention mechanism-based feature extraction. For the different imaging principles between the depth map and the RGB image, the correspondence between two kinds of features cannot be guaranteed.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Tan,

Zhang,

Teng

et al. 2024

Sensors

View full text Add to dashboard Cite

Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the targetless calibrating methods in LiDAR and camera calibration. Due to its advantages of fast speed, high accuracy, and robustness under complex conditions, it has gradually been applied in practice from a simple theoretical model in just a few years, becoming an indispensable and important method. This paper systematically summarizes the research and development of this type of method in recent years. According to the principle of calibration parameter estimation, learning-based calibration algorithms are divided into two categories: accurate calibrating estimation and relative calibrating prediction. The evolution routes and algorithm frameworks of these two types of algorithms are elaborated, and the methods used in the algorithms’ steps are summarized. The algorithm mechanism, advantages, limitations, and applicable scenarios are discussed. Finally, we make a summary, pointing out existing research issues and trends for future development.

show abstract

Section: Bf Depth Disparitymentioning

confidence: 99%

mentioning

confidence: 99%

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Tan,

Zhang,

Teng

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Zhang,

Ma,

Wang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

As the combination of panoramic images and laser point clouds becomes more and more widely used as a technique, the accurate determination of external parameters has become essential. However, due to the relative position change of the sensor and the time synchronization error, the automatic and accurate matching of the panoramic image and the point cloud is very challenging. In order to solve this problem, this paper proposes an automatic and accurate registration method for panoramic images and point clouds of urban large scenes based on line features. Firstly, the multi-modal point cloud line feature extraction algorithm is used to extract the edge of the point cloud. Based on the point cloud intensity orthoimage (an orthogonal image based on the point cloud’s intensity values), the edge of the road markings is extracted, and the geometric feature edge is extracted by the 3D voxel method. Using the established virtual projection correspondence for the panoramic image, the panoramic image is projected onto the virtual plane for edge extraction. Secondly, the accurate matching relationship is constructed by using the feature constraint of the direction vector, and the edge features from both sensors are refined and aligned to realize the accurate calculation of the registration parameters. The experimental results show that the proposed method shows excellent registration results in challenging urban scenes. The average registration error is better than 3 pixels, and the root mean square error (RMSE) is less than 1.4 pixels. Compared with the mainstream methods, it has advantages and can promote the further research and application of panoramic images and laser point clouds.

show abstract

End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles

Cited by 2 publications

References 22 publications

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Contact Info

Product

Resources

About