Vision based vehicle relocalization in 3D line-feature map using Perspective-n-Line with a known vertical direction

Abstract

Common approaches for vehicle localization propose to match LiDAR data or 2D features from cameras to a prior 3D LiDAR map. Yet, these methods require both heavy computational power often provided by GPU, and a first rough localization estimate via GNSS to be performed online. Moreover, storing and accessing 3D dense LiDAR maps can be challenging in case of city-wide coverage. In this paper, we address the problem of camera global relocalization in a prior 3D line-feature map from a single image, in a GNSS denied context and with no prior pose estimation. We propose a dual contribution. (1) We introduce a novel pose estimation method from lines, (i.e. Perspective-n-Line or PnL), with a known vertical direction. Our method benefits a Gauss-Newton optimization scheme to compensate the sensor-induced vertical direction errors, and refine the overall pose. Our algorithm requires at least 3 lines to output a pose (P3L) and requires no reformulation to operate with a higher number of lines. (2) We propose a RANSAC (RANdom SAmple Consensus) 2D-3D line matching and outliers removal algorithm requiring solely one 2D-3D line pair to operate, i.e. RANSAC1. Our method reduces the number of iteration required to match features and can be easily modified to exhaustively test all feature combinations. We evaluate the robustness of our algorithms with a synthetic data, and on a challenging sub-sequence of the KITTI dataset. *This work is part of the COPTER project, and is co-funded by the European Union and the Région Normandie. Europe is involved in Normandy through the European Regional Development Fund (ERDF)

Trifocal Tensor and Relative Pose Estimation from 8 Lines and Known Vertical Direction

“…A set of random 2D pairs of points is generated on the image planes. Each pair defines a 2D line which is at least 70 pixel long [59].…”

“…The KITTI dataset provides ground truth poses for the sequences, which is directly provided from the output of the built-in GPS/IMU units. A challenging subsequence with two consecutive sharp turns are selected, which starts from image 1223 to image 1276 in the sequence 00 [59]. These 54 images are taken place in an urban environment, which contains a large number of line features.…”

Pose and Focal Length Estimation Using Two Vanishing Points with Known Camera Position

“…Simultaneously, the computational complexity changes from the nonlinear solution of the P3L methods to a linear solution, which can improve the computational speed and accuracy. In addition, some methods use sensors to measure the partial pose information in advance, such as vertical direction [ 36 , 37 ] or camera position [ 38 ], to reduce the number of 2D–3D line correspondences required and improve the accuracy and computational speed. Furthermore, without reducing the number of 2D–3D line correspondences, some intrinsic parameters, such as focal length, can be simultaneously estimated.…”

Robot arm gripping pose estimation algorithm based on binocular camera

“…This method has limitations when the 3D line and 3D model are unknown. Lecrosnier L. et al [17] proposed a pose estimation method based on PNL algorithm and a matching/outlier removal method based on RANSAC algorithm and dependent on vertical direction to locate cars in 3D line maps, so as to improve the calculation accuracy and reduce the number of iterations. The interference of light and noise to line features is less than that of point features, so the method based on line features is suitable for weakly textured objects with smooth surface and few feature points.…”