SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding

Bujanca, Mihai; Gafton, Paul; Saeedi, Sajad; Nisbet, Andy; Bodin, Bruno; Michael, F P OaBoyle; Davison, Andrew J.; Paul, H J Kelly; Riley, Graham; Lennox, Barry; Luján, Mikel; Furber, Steve

doi:10.1109/icra.2019.8794369

Cited by 31 publications

(15 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate a fusion model, it is necessary to establish an evaluation method for a multi-modal sensor fusion model [ 145 ]. In spite of several mature evaluation methods, such as the Monte Carlo strategy, real-time simulation, and individual calculation in the specific applications, and some SLAM evaluation methods, for instance relative pose error (RPE) [ 146 ], absolute trajectory error (ATE) [ 146 ], etc., they are not suitable to evaluate the performance of each sensor in the whole fusion system.…”

Section: Discussion and Future Trendsmentioning

confidence: 99%

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

Yang

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

Indoor autonomous navigation refers to the perception and exploration abilities of mobile agents in unknown indoor environments with the help of various sensors. It is the basic and one of the most important functions of mobile agents. In spite of the high performance of the single-sensor navigation method, multi-sensor fusion methods still potentially improve the perception and navigation abilities of mobile agents. This work summarizes the multi-sensor fusion methods for mobile agents’ navigation by: (1) analyzing and comparing the advantages and disadvantages of a single sensor in the task of navigation; (2) introducing the mainstream technologies of multi-sensor fusion methods, including various combinations of sensors and several widely recognized multi-modal sensor datasets. Finally, we discuss the possible technique trends of multi-sensor fusion methods, especially its technique challenges in practical navigation environments.

show abstract

Section: Discussion and Future Trendsmentioning

confidence: 99%

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

Yang

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…In future, we plan to apply the metric to other SLAM datasets, so the that developers can selectively work with trajectories with known difficulty levels. We also plan to add the metric to SLAMBench3.0 [4], and extended the metric to support multi-robot trajectories [21].…”

Section: Discussionmentioning

confidence: 99%

“…In SLAMBench, a SLAM algorithm (specifically KinectFusion [2]) is measured in terms of both accuracy and computational cost across a range of processor platforms and using different language implementations. In SLAMBench2.0 [3] and SLAMBench3.0 [4], more SLAM algorithms are supported by a SLAM API and an I/O system. Similar benchmarking works have been performed in [5] and [6].…”

Section: Introductionmentioning

confidence: 99%

Characterizing Visual Localization and Mapping Datasets

Saeedi

Carvalho

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Benchmarking mapping and motion estimation algorithms is established practice in robotics and computer vision. As the diversity of datasets increases, in terms of the trajectories, models, and scenes, it becomes a challenge to select datasets for a given benchmarking purpose. Inspired by the Wasserstein distance, this paper addresses this concern by developing novel metrics to evaluate trajectories and the environments without relying on any SLAM or motion estimation algorithm. The metrics, which so far have been missing in the research community, can be applied to the plethora of datasets that exist. Additionally, to improve the robotics SLAM benchmarking, the paper presents a new dataset for visual localization and mapping algorithms. A broad range of real-world trajectories is used in very high-quality scenes and a rendering framework to create a set of synthetic datasets with groundtruth trajectory and dense map which are representative of key SLAM applications such as virtual reality (VR), micro aerial vehicle (MAV) flight, and ground robotics.

show abstract

“…We evaluate our implementation using the SLAMBench framework [3] [4]. All experiments were performed on a machine with an Intel Core i7-6700HQ CPU with 16GB of memory, and an NVidia GeForce GTX 960M with 4GB VRAM, running Ubuntu 18.04.…”

Section: Methodsmentioning

confidence: 99%

“…Finally, we see our work in the context of emerging technologies for benchmarking (e.g SLAMBench [4] and hyperparameter tuning (e.g HyperMapper [2]), which provide opportunities to tailor SLAM systems to suit specific applications. In this respect, FullFusion lays groundwork towards finding the right combination of systems for dynamic environments.…”

Section: Limitations and Future Workmentioning

confidence: 99%

FullFusion: A Framework for Semantic Reconstruction of Dynamic Scenes

Bujanca

Luján

Lennox

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

Assuming that scenes are static is common in SLAM research. However, the world is complex, dynamic, and features interactive agents. Mobile robots operating in a variety of environments in real-life scenarios require an advanced level of understanding of their surroundings. Therefore, it is crucial to find effective ways of representing the world in its dynamic complexity, beyond the geometry of static scene elements. We present a framework that enables incremental reconstruction of semantically-annotated 3D models in dynamic settings using commodity RGB-D sensors. Our method is the first to perform semantic reconstruction of non-rigidly deforming objects along with a static background. FullFusion is a step towards enabling robots to have a deeper and richer understanding of their surroundings, and can facilitate the study of interaction and scene dynamics. To showcase the potential of FullFusion, we provide a quantitative and qualitative evaluation on a baseline implementation which employs specific reconstruction and segmentation pipelines. It is, however, important to highlight that the modular design of the framework allows us to easily replace any of the components with new or existing counterparts.

show abstract

SLAMBench 3.0: Systematic Automated Reproducible Evaluation of SLAM Systems for Robot Vision Challenges and Scene Understanding

Abstract: The first two authors have equal contribution, the order just reflects alphabetical order.

Cited by 31 publications

References 60 publications

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

An Outline of Multi-Sensor Fusion Methods for Mobile Agents Indoor Navigation

Characterizing Visual Localization and Mapping Datasets

FullFusion: A Framework for Semantic Reconstruction of Dynamic Scenes

Contact Info

Product

Resources

About