Semantics for Robotic Mapping, Perception and Interaction: A Survey

Garg, Sourav; Sünderhauf, Niko; Dayoub, Feras; Morrison, Douglas; Cosgun, Akansel; Carneiro, Gustavo; Wu, Qi; Chin, Tat-Jun; Reid, Ian; Gould, Stephen J.; Corke, Peter; Milford, Michael

doi:10.1561/2300000059

Cited by 88 publications

(31 citation statements)

References 807 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other Interesting Approaches to VPR: Other interesting approaches to place recognition include semanticsegmentation-based VPR (as in Arandjelović and Zisserman 2014b;Mousavian et al 2015;Stenborg et al 2018;Schönberger et al 2018;Naseer et al 2017) and objectproposals-based VPR (Hou et al 2018), as recently reviewed by Garg et al (2020). For images containing repetitive structures, Torii et al (2013) proposed a robust mechanism for collecting visual words into descriptors.…”

Section: Literature Reviewmentioning

confidence: 99%

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

et al. 2021

Self Cite

View full text Add to dashboard Cite

Visual place recognition (VPR) is the process of recognising a previously visited place using visual information, often under varying appearance conditions and viewpoint changes and with computational constraints. VPR is related to the concepts of localisation, loop closure, image retrieval and is a critical component of many autonomous navigation systems ranging from autonomous vehicles to drones and computer vision systems. While the concept of place recognition has been around for many years, VPR research has grown rapidly as a field over the past decade due to improving camera hardware and its potential for deep learning-based techniques, and has become a widely studied topic in both the computer vision and robotics communities. This growth however has led to fragmentation and a lack of standardisation in the field, especially concerning performance evaluation. Moreover, the notion of viewpoint and illumination invariance of VPR techniques has largely been assessed qualitatively and hence ambiguously in the past. In this paper, we address these gaps through a new comprehensive open-source framework for assessing the performance of VPR techniques, dubbed “VPR-Bench”. VPR-Bench (Open-sourced at: https://github.com/MubarizZaffar/VPR-Bench) introduces two much-needed capabilities for VPR researchers: firstly, it contains a benchmark of 12 fully-integrated datasets and 10 VPR techniques, and secondly, it integrates a comprehensive variation-quantified dataset for quantifying viewpoint and illumination invariance. We apply and analyse popular evaluation metrics for VPR from both the computer vision and robotics communities, and discuss how these different metrics complement and/or replace each other, depending upon the underlying applications and system requirements. Our analysis reveals that no universal SOTA VPR technique exists, since: (a) state-of-the-art (SOTA) performance is achieved by 8 out of the 10 techniques on at least one dataset, (b) SOTA technique in one community does not necessarily yield SOTA performance in the other given the differences in datasets and metrics. Furthermore, we identify key open challenges since: (c) all 10 techniques suffer greatly in perceptually-aliased and less-structured environments, (d) all techniques suffer from viewpoint variance where lateral change has less effect than 3D change, and (e) directional illumination change has more adverse effects on matching confidence than uniform illumination change. We also present detailed meta-analyses regarding the roles of varying ground-truths, platforms, application requirements and technique parameters. Finally, VPR-Bench provides a unified implementation to deploy these VPR techniques, metrics and datasets, and is extensible through templates.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Enriched Reference Maps: With the rapid increase in data gathering, more so in the field of autonomous driving, it is high time to consider the use of an enriched reference map, which could be in the form of multiple reference images per location [Churchill and Newman, 2012] or semanticallyannotated 3D maps [Garg et al, 2020b]. In the simplest case, choosing the best reference set can lead to vast performance improvements.…”

Section: Place Representationmentioning

confidence: 99%

Where Is Your Place, Visual Place Recognition?

Garg

Fischer

Milford

2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Visual Place Recognition (VPR) is often characterized as being able to recognize the same place despite significant changes in appearance and viewpoint. VPR is a key component of Spatial Artificial Intelligence, enabling robotic platforms and intelligent augmentation platforms such as augmented reality devices to perceive and understand the physical world. In this paper, we observe that there are three "drivers" that impose requirements on spatially intelligent agents and thus VPR systems: 1) the particular agent including its sensors and computational resources, 2) the operating environment of this agent, and 3) the specific task that the artificial agent carries out. In this paper, we characterize and survey key works in the VPR area considering those drivers, including their place representation and place matching choices. We also provide a new definition of VPR based on the visual overlap - akin to spatial view cells in the brain - that enables us to find similarities and differences to other research areas in the robotics and computer vision fields. We identify several open challenges and suggest areas that require more in-depth attention in future works.

show abstract

“…Loop closure technique can mitigate the cumulative error and appearance-based loop-closure detection [4] has been widely used. At the same time, the use of semantic information on place recognition has attracted a lot of interest recently [11], while considering that a false positive loop closure could highly downgrade and even corrupt the back-end optimization process of SLAM systems [12][13][14], it is of great value to effectively identify and reject false loop closures to obtain more accurate positioning results and more robust positioning performance both for single-robot and multi-robot collaborative SLAM systems. The aforementioned objective could be realized through improving the accuracy of place recognition, for example by using deep learning or other novel technologies, but it should be pointed out that mismatches are almost inevitable, especially in large scenarios involving collaborative SLAM systems.…”

Section: Related Workmentioning

confidence: 99%

Using a Two-Stage Method to Reject False Loop Closures and Improve the Accuracy of Collaborative SLAM Systems

Zhang

Wang

et al. 2021

Electronics

View full text Add to dashboard Cite

Loop-closure detection is an essential means to reduce accumulated errors of simultaneous localization and mapping (SLAM) systems. However, even false positive loop closures could seriously interfere and even corrupt the back-end optimization process. For a collaborative SLAM system that generally uses both intra-robot and inter-robot loop closures to optimize the pose graph, it is a tough job to reject those false positive loop closures without a reliable a priori knowledge of the relative pose transformation between robots. Aiming at this solving problem, this paper proposes a two-stage false positive loop-closure rejection method based on three types of consistency checks. Firstly, a multi-robot pose-graph optimization model is given which transforms the multi-robot pose optimization problem into a maximum likelihood estimation model. Then, the principle of the false positive loop-closure rejection method based on χ2 test is proposed, in which clustering is used to reject those intra-robot false loop-closures in the first step, and a largest mutually consistent loop-based χ2 test is constructed to reject inter-robot false loop closures in the second step. Finally, an open dataset and synthetic data are used to evaluate the performance of the algorithms. The experimental results demonstrate that our method improves the accuracy and robustness of the back-end pose-graph optimization with a strong ability to reject false positive loop closures, and it is not sensitive to the initial pose at the same time. In the Computer Science and Artificial Intelligence Lab (CSAIL) dataset, the absolute position error is reduced by 55.37% compared to the dynamic scaling covariance method, and the absolute rotation error is reduced by 77.27%; in the city10,000 synthetic dataset, the absolute position error is reduced by 89.37% compared to the pairwise consistency maximization (PCM) and the absolute rotation error is reduced by 97.9%.

show abstract

Semantics for Robotic Mapping, Perception and Interaction: A Survey

Cited by 88 publications

References 807 publications

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

Where Is Your Place, Visual Place Recognition?

Using a Two-Stage Method to Reject False Loop Closures and Improve the Accuracy of Collaborative SLAM Systems

Contact Info

Product

Resources

About