Targetless Rotational Auto-Calibration of Radar and Camera for Intelligent Transportation Systems

Abstract

Most intelligent transportation systems use a combination of radar sensors and cameras for robust vehicle perception. The calibration of these heterogeneous sensor types in an automatic fashion during system operation is challenging due to differing physical measurement principles and the high sparsity of traffic radars. We propose -to the best of our knowledge -the first data-driven method for automatic rotational radar-camera calibration without dedicated calibration targets. Our approach is based on a coarse and a fine convolutional neural network. We employ a boosting-inspired training algorithm, where we train the fine network on the residual error of the coarse network. Due to the unavailability of public datasets combining radar and camera measurements, we recorded our own real-world data. We demonstrate that our method is able to reach precise and robust sensor registration and show its generalization capabilities to different sensor alignments and perspectives.

Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review

“…The studies of extrinsic calibration and the methodologies are well-established in the literature, see reference [ 143 , 144 , 145 , 146 , 147 , 148 , 149 , 150 , 151 ] for example. Though, the extrinsic calibration of multiple sensors with various physical measurement principles can pose a challenge in multi-sensor systems.…”

“…Though, the extrinsic calibration of multiple sensors with various physical measurement principles can pose a challenge in multi-sensor systems. For instance, it is often challenging to match the corresponding features between camera images (dense data in pixels) and 3D LiDAR or radar point clouds (sparse depth data without color information) [ 144 ]. The target-based extrinsic calibration approach employs specially designed calibration targets, such as marker-less planar pattern [ 51 ], checkerboard pattern [ 145 ], orthogonal and trihedral reflector [ 51 , 143 , 146 , 148 ], circular pattern to calibrate multiple sensor modalities in autonomous systems.…”

“…The targetless extrinsic calibration approach leverages the estimated motion by individual sensors or utilizes the features in the perceiving environment to calibrate the sensors. However, employing the perceived environment features requires multimodal sensors to extract the same features within the environment and is sensitive to the calibration environment [ 144 , 149 ].…”

Robust Target Detection and Tracking Algorithm Based on Roadside Radar and Camera

“…In order to improve the robustness of RPU, some scholars have proposed methods based on radar and camera fusion for vehicle detection and width estimation in bad weather [ 36 , 37 ]. Christoph S. et al proposed a two-stream CNN method for auto-calibration of a radar and camera to realize robust detection of vehicles on the highway [ 38 ]. Kaul P. at al.…”

Estimation of Vehicle Trailer Rotation Using Calibrated Radars With Multiple Viewpoints

“…Some existing calibration methods have been proposed for different sensors; for cameras in [38,39], for lidar and camera in [40][41][42], for laser scanner and camera in [43][44][45][46], for radar and camera in [63][64][65], and for radar, lidar, and camera in [66,67].…”