Fast and Memory Efficient Graph Optimization via ICM for Visual Place Recognition

Schubert, Stefan; Neubert, Peer; Protzel, Peter

doi:10.15607/rss.2021.xvii.091

Cited by 13 publications

(19 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, starting around 2012, an increasing number of works explored VPR under severe condition changes, e.g., due to the day-night cycle or seasonal changes [14]. This shift can also be seen by the emergence of datasets with condition changes that have appeared since 2012 [1], [3]. In 2014, the use of deep learning for VPR [15] emerged as a way to handle challenging data and has since proven effective in changing environments [16].…”

Section: B the Vpr Problem And Its Details As Reflected In This Tutorialmentioning

confidence: 99%

Visual Place Recognition: A Tutorial

Schubert¹,

Neubert²,

Garg³

et al. 2023

Preprint

View full text Add to dashboard Cite

is an essential capability for mobile robots, enabling them to build a comprehensive representation of their environment and interact with the environment effectively towards a goal. A rapidly growing field of research in this area is Visual Place Recognition (VPR), which is the ability to recognize previously seen places in the world based solely on images. The volume of published research on VPR has shown a significant and continuous growth over the years, from two papers with "visual place recognition" and seven papers with "place recognition" in the title in 2006, compared to 65 and 163 papers, respectively, in 2022 1 . A number of survey and benchmarking papers have discussed the challenges, open questions, and achievements in the field of VPR [1], [2], [3], [4], [5], [6].This present work is the first tutorial paper on visual place recognition. It unifies the terminology of VPR and complements prior research in two important directions:1) It provides a systematic introduction for newcomers to the field, covering topics such as the formulation of the VPR problem, a generic algorithmic pipeline, an evaluation methodology for VPR approaches, and the major challenges for VPR and how they may be addressed. 2) As a contribution for researchers acquainted with the VPR problem, it examines the intricacies of different VPR problem types regarding input (database or query set), data processing (online or offline) and output (one or multiple matches per query image). The tutorial also discusses the subtleties behind the evaluation of VPR algorithms, e.g., the evaluation of a VPR system that has to find all matching database images per query, as opposed to just a single match.

show abstract

Section: B the Vpr Problem And Its Details As Reflected In This Tutorialmentioning

confidence: 99%

Visual Place Recognition: A Tutorial

Schubert¹,

Neubert²,

Garg³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Established approaches for visual loop closure detection in robotics trace back to place recognition and image retrieval techniques in computer vision; these approaches are broadly adopted in SLAM pipelines, but are known to suffer from appearance and viewpoint changes [36]. Recent approaches investigate place recognition using image sequences [20,55] or deep learning [3]. More related to our proposal is the set of papers leveraging semantic information for loop closure detection.…”

Section: Related Workmentioning

confidence: 99%

Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization

Hughes¹,

Chang²,

Carlone³

2022

Preprint

View full text Add to dashboard Cite

3D scene graphs have recently emerged as a powerful high-level representation of 3D environments. A 3D scene graph describes the environment as a layered graph where nodes represent spatial concepts at multiple levels of abstraction (from low-level geometry to high-level semantics including objects, places, rooms, buildings, etc.) and edges represent relations between concepts. While 3D scene graphs can serve as an advanced "mental model" for robots, how to build such a rich representation in real-time is still uncharted territory.This paper describes the first real-time Spatial Perception engINe (SPIN), a suite of algorithms to build a 3D scene graph from sensor data in real-time. Our first contribution is to develop real-time algorithms to incrementally construct the layers of a scene graph as the robot explores the environment; these algorithms build a local Euclidean Signed Distance Function (ESDF) around the current robot location, extract a topological map of places from the ESDF, and then segment the places into rooms using an approach inspired by community-detection techniques. Our second contribution is to investigate loop closure detection and optimization in 3D scene graphs. We show that 3D scene graphs allow defining hierarchical descriptors for loop closure detection; our descriptors capture statistics across layers in the scene graph, ranging from low-level visual appearance, to summary statistics about objects and places. We then propose the first algorithm to optimize a 3D scene graph in response to loop closures; our approach relies on embedded deformation graphs to simultaneously correct all layers of the scene graph. We implement the proposed SPIN into a highly parallelized architecture, named Hydra, that combines fast early and midlevel perception processes (e.g., local mapping) with slower highlevel perception (e.g., global optimization of the scene graph). We evaluate Hydra on simulated and real data and show it is able to reconstruct 3D scene graphs with an accuracy comparable with batch offline methods, while running online.

show abstract

“…To integrate these image retrieval methods into the robotics context of place recognition, there is a wide range of methods, e.g. [26,35,29], that exploit the structure of the robotics task or additional available information about the database and query set for performance improvements. For example, SeqSLAM [26] exploits sequences in the database and query set and ICM [35] additionally leverages descriptor similarities within these sets.…”

Section: Related Workmentioning

confidence: 99%

SEER: Unsupervised and sample-efficient environment specialization of image descriptors

Neubert¹,

Schubert²

2022

Robotics: Science and Systems XVIII

Self Cite

View full text Add to dashboard Cite

Image descriptor based place recognition is an important means for loop-closure detection in SLAM. The currently best performing image descriptors for this task are trained on large training datasets with the goal to be applicable in many different environments. In particular, they are not optimized for a specific environment, e.g. the city of Oxford. However, we argue that for place recognition, there is always a specific environment -not necessarily geographically defined, but specified by the particular set of descriptors in the database. In this paper, we propose SEER, a simple and efficient algorithm that can learn to create better descriptors for a specific environment from such a potentially very small set of database descriptors. The new descriptors are better in the sense that they will be more suited for image retrieval on these database descriptors. SEER stands for Sparse Exemplar Ensemble Representations. Both sparsity and ensemble representations are necessary components of the proposed approach. This is evaluated on a large variety of standard place recognition datasets where SEER considerably outperforms existing methods. It does not require any label information and is applicable in online place recognition scenarios. Open source code is available. 1

show abstract

Fast and Memory Efficient Graph Optimization via ICM for Visual Place Recognition

Cited by 13 publications

References 30 publications

Visual Place Recognition: A Tutorial

Visual Place Recognition: A Tutorial

Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization

SEER: Unsupervised and sample-efficient environment specialization of image descriptors

Contact Info

Product

Resources

About