Semantic Mapping Based on Spatial Concepts for Grounding Words Related to Places in Daily Environments

Abstract

An autonomous robot performing tasks in a human environment needs to recognize semantic information about places. Semantic mapping is a task in which suitable semantic information is assigned to an environmental map so that a robot can communicate with people and appropriately perform tasks requested by its users. We propose a novel statistical semantic mapping method called SpCoMapping, which integrates probabilistic spatial concept acquisition based on multimodal sensor information and a Markov random field applied for learning the arbitrary shape of a place on a map.SpCoMapping can connect multiple words to a place in a semantic mapping process using user utterances without pre-setting the list of place names. We also develop a nonparametric Bayesian extension of SpCoMapping that can automatically estimate an adequate number of categories. In the experiment in the simulation environments, we showed that the proposed method generated better semantic maps than previous semantic mapping methods; our semantic maps have categories and shapes similar to the ground truth provided by the user. In addition, we showed that SpCoMapping could generate appropriate semantic maps in a real-world environment.

Spatial concept-based navigation with human speech instructions via probabilistic inference on Bayesian generative model

“…This case occurred owing to the fact that the range of the learned spatial concept exceeded the wall. This issue occurs because the position distribution of the spatial concept is represented by a Gaussian distribution; however, it is solved via application of an approach that deals with the shape of the room based on the conditional random field in [36].…”

“…Our method has the advantage that it can be also applied to different POMDP-based models for spatial concepts, e.g. [36,37]. SpCoNavi was assumed to use parameters estimated by SpCoSLAM.…”

Semiotically adaptive cognition: toward the realization of remotely-operated service robots for the new normal symbiotic society

“…This can accelerate the learning process in a newly visited environment [36]. Katsumata et al employed generative adversarial networks (GAN) that model complex knowledge on semantic mapping across many home environments [37]. The development of a general framework to combine common-sense symbolic knowledge and local knowledge acquired onsite and online is also an important challenge.…”

Interactive Natural Language Grounding via Referring Expression Comprehension and Scene Graph Parsing

“…Patki et al ( 2019 ) utilized distributed correspondence graph to infer the environment representation in a task-specific approach. Katsumata et al ( 2019 ) introduced a statistical semantic mapping method that enables the robot to connect multiple words embedded in spoken utterance to a place in a semantic mapping processing. However, these models did not take into account the inherent vagueness of natural language.…”

“…Natural language grounding-based HRI requires a comprehensive understanding of natural language instructions and working scenarios, and the pivotal issue of is to locate the referred objects in working scenarios according to given instructions. Although the existing models achieve promising results, some of them either do not take the inherent ambiguity of natural language into consideration (Paul et al, 2018 ; Katsumata et al, 2019 ; Mi et al, 2019 ; Patki et al, 2019 ), or alleviate the ambiguity via drawing support from auxiliary information, such as dialogue system (Ahn et al, 2018 ; Hatori et al, 2018 ; Shridhar and Hsu, 2018 ) and gestures (Shridhar and Hsu, 2018 ). However, the dialogue-based disambiguation systems entail time cost and cumbersome interactions.…”