Global visual localization in LiDAR-maps through shared 2D-3D embedding space

Cattaneo, Daniele; Vaghi, Matteo; Fontana, Simone; Ballardini, Augusto Luis; Sorrenti, Domenico G.

doi:10.1109/icra40945.2020.9196859

Cited by 41 publications

(15 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While for place recognition, there are few methods performed on heterogeneous measurements. Cattaneo et al (2020) built shared embedding space for visual and lidar, thus achieving global visual localization on lidar maps via place recognition. Some researchers proposed to conduct the fusion of image and lidar points for place recognition (Xie et al, 2020).…”

Section: Multi-modal Measurements For Robotic Perceptionmentioning

confidence: 99%

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Yin

Wang

et al. 2021

Front. Robot. AI

View full text Add to dashboard Cite

Place recognition is critical for both offline mapping and online localization. However, current single-sensor based place recognition still remains challenging in adverse conditions. In this paper, a heterogeneous measurement based framework is proposed for long-term place recognition, which retrieves the query radar scans from the existing lidar (Light Detection and Ranging) maps. To achieve this, a deep neural network is built with joint training in the learning stage, and then in the testing stage, shared embeddings of radar and lidar are extracted for heterogeneous place recognition. To validate the effectiveness of the proposed method, we conducted tests and generalization experiments on the multi-session public datasets and compared them to other competitive methods. The experimental results indicate that our model is able to perform multiple place recognitions: lidar-to-lidar (L2L), radar-to-radar (R2R), and radar-to-lidar (R2L), while the learned model is trained only once. We also release the source code publicly: https://github.com/ZJUYH/radar-to-lidar-place-recognition.

show abstract

Section: Multi-modal Measurements For Robotic Perceptionmentioning

confidence: 99%

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Yin

Wang

et al. 2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…The central insight besides the Teacher/Student paradigm is to learn a shared embedding space between the 2D images created with the intersection model and those generated by the aforementioned transformation pipelines. This approach is inspired by the work of Cattaneo et al [ 55 ], which performs visual localization using 2D and 3D inputs in a bi-directional mode, teaching two networks to create a shared embedding space and thus enabling a two-way localization, starting either from 2D or 3D inputs. Recalling the metric technique described in the first part of Section 4.2 , the teacher-student paradigm introduced some minor changes, in particular to Equation ( 1 ).…”

Section: Technical Approachmentioning

confidence: 99%

Urban Intersection Classification: A Comparative Analysis

Ballardini

Hernández

Carrasco

et al. 2021

Sensors

View full text Add to dashboard Cite

Understanding the scene in front of a vehicle is crucial for self-driving vehicles and Advanced Driver Assistance Systems, and in urban scenarios, intersection areas are one of the most critical, concentrating between 20% to 25% of road fatalities. This research presents a thorough investigation on the detection and classification of urban intersections as seen from onboard front-facing cameras. Different methodologies aimed at classifying intersection geometries have been assessed to provide a comprehensive evaluation of state-of-the-art techniques based on Deep Neural Network (DNN) approaches, including single-frame approaches and temporal integration schemes. A detailed analysis of most popular datasets previously used for the application together with a comparison with ad hoc recorded sequences revealed that the performances strongly depend on the field of view of the camera rather than other characteristics or temporal-integrating techniques. Due to the scarcity of training data, a new dataset is created by performing data augmentation from real-world data through a Generative Adversarial Network (GAN) to increase generalizability as well as to test the influence of data quality. Despite being in the relatively early stages, mainly due to the lack of intersection datasets oriented to the problem, an extensive experimental activity has been performed to analyze the individual performance of each proposed systems.

show abstract

“…In order to compare the images generated from the intersection model and those transformed from the RGB images, we propose a teacher/student paradigm aimed to learn a shared-embedding space between the two domains. The approach proposed in this work is inspired by the works of Cattaneo [22], which performs visual localization using 2D and 3D inputs in a bi-directional mode, teaching two networks to create a shared embedding space. In a similar way, we conceive the classification problem as a metric-learning task where, given two instances of the same intersections class but in different domains, e.g., Class 0 and Domains D1 and D2 (D 1 C=0 and D 2 C=0 ), and two different non-linear functions f (•) and g(•) represented in form of DNNs, the distance between the embeddings is lower than any other negative intersection instance, e.g., D 2 c=2 .…”

Section: Teacher/student Trainingmentioning

confidence: 99%

Model Guided Road Intersection Classification

Ballardini,

Hernández,

Sotelo

2021

Preprint

View full text Add to dashboard Cite

Understanding complex scenarios from in-vehicle cameras is essential for safely operating autonomous driving systems in densely populated areas. Among these, intersection areas are one of the most critical as they concentrate a considerable number of traffic accidents and fatalities. Detecting and understanding the scene configuration of these usually crowded areas is then of extreme importance for both autonomous vehicles and modern Advanced Driver Assistance Systems (ADAS) aimed at preventing road crashes and increasing the safety of vulnerable road users. This work investigates intersection classification from RGB images using well-consolidate neural network approaches along with a method to enhance the results based on the teacher/student training paradigm. An extensive experimental activity aimed at identifying the best input configuration and evaluating different network parameters on both the well-known KITTI dataset and the new KITTI-360 sequences shows that our method outperforms current state-of-the-art approaches on a per-frame basis and prove the effectiveness of the proposed learning scheme.

show abstract

Global visual localization in LiDAR-maps through shared 2D-3D embedding space

Cited by 41 publications

References 33 publications

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Radar-to-Lidar: Heterogeneous Place Recognition via Joint Learning

Urban Intersection Classification: A Comparative Analysis

Model Guided Road Intersection Classification

Contact Info

Product

Resources

About