Automatic Targetless Monocular Camera and LiDAR External Parameter Calibration Method for Mobile Robots

Yu, Ying; Fan, Song; Li, Lei; Wang, Tao; Li, Li

doi:10.3390/rs15235560

Cited by 3 publications

(2 citation statements)

References 46 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhengyou Zhang's approach strikes a balance, offering simple yet mature technology. Utilizing a fixed checkerboard grid during image acquisition at various angles and positions, the calibration process establishes equations based on key points, with parameter values determined through maximum likelihood estimation [17][18][19]. In this paper, Zhengyou Zhang's method, implemented with MATLAB calibration toolbox, is employed for offline calibration to find the camera's internal reference, external reference, and distortion parameters.…”

Section: Camera Calibrationmentioning

confidence: 99%

Research on Target Ranging Method for Live-Line Working Robots

Hua,

Chen,

Luo

et al. 2024

Symmetry

View full text Add to dashboard Cite

Due to the operation of live-line working robots at elevated heights for precision tasks, a suitable visual assistance system is essential to determine the position and distance of the robotic arm or gripper relative to the target object. In this study, we propose a method for distance measurement in live-line working robots by integrating the YOLOv5 algorithm with binocular stereo vision. The camera’s intrinsic and extrinsic parameters, as well as distortion coefficients, are obtained using the Zhang Zhengyou calibration method. Subsequently, stereo rectification is performed on the images to establish a standardized binocular stereovision model. The Census and Sum of Absolute Differences (SAD) fused stereo matching algorithm is applied to compute the disparity map. We train a dataset of transmission line bolts within the YOLO framework to derive the optimal model. The identified bolts are framed, and the depth distance of the target is ultimately calculated. And through the experimental verification of the bolt positioning, the results show that the method can achieve a relative error of 1% in the proximity of positioning. This approach provides real-time and accurate environmental perception for symmetrical structural live-line working robots, enhancing the stability of these robots.

show abstract

Section: Camera Calibrationmentioning

confidence: 99%

Research on Target Ranging Method for Live-Line Working Robots

Hua,

Chen,

Luo

et al. 2024

Symmetry

View full text Add to dashboard Cite

show abstract

“…One significant advantage of LiDAR in 3D object detection is its ability to capture fine-grained details of objects, such as their shape, size, spatial position, and corresponding intensity information. These detailed pieces of information enable precise identification and tracking of objects, even in challenging scenarios characterized by complex environments [16][17][18][19][20][21][22][23][24]. Furthermore, LiDAR point cloud data are typically less affected by environmental conditions such as lighting variations, making them highly reliable for 3D object detection tasks.…”

Section: Introductionmentioning

confidence: 99%

Equal Emphasis on Data and Network: A Two-Stage 3D Point Cloud Object Detection Algorithm with Feature Alignment

Xiao,

Li,

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Three-dimensional object detection is a pivotal research topic in computer vision, aiming to identify and locate objects in three-dimensional space. It has wide applications in various fields such as geoscience, autonomous driving, and drone navigation. The rapid development of deep learning techniques has led to significant advancements in 3D object detection. However, with the increasing complexity of applications, 3D object detection faces a series of challenges such as data imbalance and the effectiveness of network models. Specifically, in an experiment, our investigation revealed a notable discrepancy in the LiDAR reflection intensity within a point cloud scene, with stronger intensities observed in proximity and weaker intensities observed at a distance. Furthermore, we have also noted a substantial disparity in the number of foreground points compared to the number of background points. Especially in 3D object detection, the foreground point is more important than the background point, but it is usually downsampled without discrimination in the subsequent processing. With the objective of tackling these challenges, we work from both data and network perspectives, designing a feature alignment filtering algorithm and a two-stage 3D object detection network. Firstly, in order to achieve feature alignment, we introduce a correction equation to decouple the relationship between distance and intensity and eliminate the attenuation effect of intensity caused by distance. Then, a background point filtering algorithm is designed by using the aligned data to alleviate the problem of data imbalance. At the same time, we take into consideration the fact that the accuracy of semantic segmentation plays a crucial role in 3D object detection. Therefore, we propose a two-stage deep learning network that integrates spatial and spectral information, in which a feature fusion branch is designed and embedded in the semantic segmentation backbone. Through a series of experiments on the KITTI dataset, it is proven that the proposed method achieves the following average precision (AP_R40) values for easy, moderate, and hard difficulties, respectively: car (Iou 0.7)—89.23%, 80.14%, and 77.89%; pedestrian (Iou 0.5)—52.32%, 45.47%, and 38.78%; and cyclist (Iou 0.5)—76.41%, 61.92%, and 56.39%. By emphasizing both data quality optimization and efficient network architecture, the performance of the proposed method is made comparable to other state-of-the-art methods.

show abstract

A LiDAR-Camera Joint Calibration Algorithm Based on Deep Learning

Ren,

Liu,

Wang

2024

Sensors

View full text Add to dashboard Cite

Multisensor (MS) data fusion is important for improving the stability of vehicle environmental perception systems. MS joint calibration is a prerequisite for the fusion of multimodality sensors. Traditional calibration methods based on calibration boards require the manual extraction of many features and manual registration, resulting in a cumbersome calibration process and significant errors. A joint calibration algorithm for a Light Laser Detection and Ranging (LiDAR) and camera is proposed based on deep learning without the need for other special calibration objects. A network model constructed based on deep learning can automatically capture object features in the environment and complete the calibration by matching and calculating object features. A mathematical model was constructed for joint LiDAR-camera calibration, and the process of sensor joint calibration was analyzed in detail. By constructing a deep-learning-based network model to determine the parameters of the rotation matrix and translation matrix, the relative spatial positions of the two sensors were determined to complete the joint calibration. The network model consists of three parts: a feature extraction module, a feature-matching module, and a feature aggregation module. The feature extraction module extracts the image features of color and depth images, the feature-matching module calculates the correlation between the two, and the feature aggregation module determines the calibration matrix parameters. The proposed algorithm was validated and tested on the KITTI-odometry dataset and compared with other advanced algorithms. The experimental results show that the average translation error of the calibration algorithm is 0.26 cm, and the average rotation error is 0.02°. The calibration error is lower than those of other advanced algorithms.

show abstract

Automatic Targetless Monocular Camera and LiDAR External Parameter Calibration Method for Mobile Robots

Cited by 3 publications

References 46 publications

Research on Target Ranging Method for Live-Line Working Robots

Research on Target Ranging Method for Live-Line Working Robots

Equal Emphasis on Data and Network: A Two-Stage 3D Point Cloud Object Detection Algorithm with Feature Alignment

A LiDAR-Camera Joint Calibration Algorithm Based on Deep Learning

Contact Info

Product

Resources

About