Semantic Monte-Carlo localization in changing environments using RGB-D cameras

Himstedt, Marian; Maehle, Erik

doi:10.1109/ecmr.2017.8098711

Cited by 8 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A VPR system is generally modeled as a ranking function, which can work with arbitrary VPR systems (e.g., Bayes filter [1], image retrieval [6], deep neural network [21]). It evaluates the likelihood of the robot being located at each predefined place class, given a query scene.…”

Section: Approachmentioning

confidence: 99%

“…In this paper, we aim to train a next-best-view (NBV) planner for active cross-domain self-localization. Given a landmark map built in a past domain (e.g., weather, season, times of the day), the goal of self-localization is to localize the robot-itself, using relative measurements from on-board landmark sensor and odometory [1]- [3]. This cross-domain self-localization problem becomes a challenging one due to appearance/removal of landmarks as well as perceptual aliasing.…”

Section: Introductionmentioning

confidence: 99%

“…Here, we propose a novel VPR-to-NBV knowledge transfer framework that uses reciprocal rank feature [12] as the dark knowledge. Specifically, our approach is based on the following two observations: (1) The environment state can be compactly represented by a local map descriptor, which is compatibile with typical input formats (e.g., image, point cloud, graph) of VPR systems [13], and (2) An arbitrary VPR system (e.g., Bayes filter [1], image retrieval [2], deep neural network [12]) can be modeled as a ranking function. The proposed approach is inspired by the recent development of transfer learning, where rank matching is used as the dark knowledge to transfer from a teacher to a student model (e.g., Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active Map-Matching: Active Self-localization by VPR-to-NBV Knowledge Transfer

Kanji¹

2021

Preprint

View full text Add to dashboard Cite

Training a next-best-view (NBV) planner for active cross-domain self-localization is an important and challenging problem. Unlike typical in-domain settings, the planner can no longer assume the environment state being constant, but must treat it as a high-dimensional component of the state variable. This study is motivated by the ability of recent visual place recognition (VPR) techniques to recognize such a high-dimensional environment state in the presence of domain-shifts. Thus, we wish to transfer the state recognition ability from VPR to NBV. However, such a VPR-to-NBV knowledge transfer is a non-trivial issue for which no known solution exists. Here, we propose to use a reciprocal rank feature, derived from the field of transfer learning, as the dark knowledge to transfer. Specifically, our approach is based on the following two observations: (1) The environment state can be compactly represented by a local map descriptor, which is compatible with typical input formats (e.g., image, point cloud, graph) of VPR systems, and (2) An arbitrary VPR system (e.g., Bayes filter, image retrieval, deep neural network) can be modeled as a ranking function. Experiments with nearest neighbor Qlearning (NNQL) show that our approach can obtain a practical NBV planner even under severe domain-shifts.

show abstract

Section: Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Active Map-Matching: Active Self-localization by VPR-to-NBV Knowledge Transfer

Kanji¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The MCL approaches also have begun to use semantic information, not only as a single landmark to localize around but also as a sensor model input. A system with an RGB-D camera was localized in a changing warehouse environment by combining the beam model using distance and bearing angle measurements and an a priori correlation table for objects [19]. The measurements were matched to an annotated grid map.…”

Section: Related Workmentioning

confidence: 99%

Robust and Efficient Indoor Localization Using Sparse Semantic Information from a Spherical Camera

Uygur

Miyagusuku

Pathak

et al. 2020

Sensors

View full text Add to dashboard Cite

Self-localization enables a system to navigate and interact with its environment. In this study, we propose a novel sparse semantic self-localization approach for robust and efficient indoor localization. “Sparse semantic” refers to the detection of sparsely distributed objects such as doors and windows. We use sparse semantic information to self-localize on a human-readable 2D annotated map in the sensor model. Thus, compared to previous works using point clouds or other dense and large data structures, our work uses a small amount of sparse semantic information, which efficiently reduces uncertainty in real-time localization. Unlike complex 3D constructions, the annotated map required by our method can be easily prepared by marking the approximate centers of the annotated objects on a 2D map. Our approach is robust to the partial obstruction of views and geometrical errors on the map. The localization is performed using low-cost lightweight sensors, an inertial measurement unit and a spherical camera. We conducted experiments to show the feasibility and robustness of our approach.

show abstract

“…Visual robot self-localization is one of most important issues in mobile robotics and has been studied in many different contexts, including multi-hypothesis pose tracking [9], map matching [10], image retrieval [11], view-sequence matching [3], etc. Our self-localization scenario is most closely related to the view-sequence matching scenario, which takes a short-term live view-sequence as query and searches for corresponding part in the map view-sequence.…”

Section: Related Workmentioning

confidence: 99%

Knowledge Transfer from Map to DNN: Use of Graph Convolutional Neural Network for Augmenting Visual Robot Self-localization System

takeda¹,

Kanji²

2020

Preprint

View full text Add to dashboard Cite

Graph-based scene model has been receiving increasing attention as a flexible and descriptive scene model for visual robot self-localization. In a typical self-localization application, objects, object features, and object relationship in an environment map are described respectively by nodes, node features, and edges in a scene graph, which are then matched against a query scene graph by a graph matching engine. However, its overhead for computation, storage, and communication, is proportional to the number and feature dimensionality of graph nodes, and can be significant in large-scale applications. In this study, we observe that graph-convolutional neural network (GCN) has a potential to become an efficient tool to train and predict with a graph matching engine. However, it is non-trivial to translate a given visual feature to a proper graph feature that contributes to good self-localization performance. To address this issue, we introduce a new knowledge transfer (KT) framework, which introduces an arbitrary self-localization model as a teacher to train the student, GCN-based self-localization system. Our KT framework enables lightweight storage/communication by using compact teacher's output signals as training data. Results on RobotCar datasets show that the proposed method outperforms existing comparing methods as well as the teacher self-localization system.

show abstract

Semantic Monte-Carlo localization in changing environments using RGB-D cameras

Cited by 8 publications

References 16 publications

Active Map-Matching: Active Self-localization by VPR-to-NBV Knowledge Transfer

Active Map-Matching: Active Self-localization by VPR-to-NBV Knowledge Transfer

Robust and Efficient Indoor Localization Using Sparse Semantic Information from a Spherical Camera

Knowledge Transfer from Map to DNN: Use of Graph Convolutional Neural Network for Augmenting Visual Robot Self-localization System

Contact Info

Product

Resources

About