Spatial concept-based navigation with human speech instructions via probabilistic inference on Bayesian generative model

Abstract

Robots are required to not only learn spatial concepts autonomously but also utilize such knowledge for various tasks in a domestic environment. Spatial concept represents a multimodal place category acquired from the robot's spatial experience including vision, speech-language, and selfposition. The aim of this study is to enable a mobile robot to perform navigational tasks with human speech instructions, such as 'Go to the kitchen', via probabilistic inference on a Bayesian generative model using spatial concepts. Specifically, path planning was formalized as the maximization of probabilistic distribution on the path-trajectory under speech instruction, based on a control-asinference framework. Furthermore, we described the relationship between probabilistic inference based on the Bayesian generative model and control problem including reinforcement learning. We demonstrated path planning based on human instruction using acquired spatial concepts to verify the usefulness of the proposed approach in the simulator and in real environments. Experimentally, places instructed by the user's speech commands showed high probability values, and the trajectory toward the target place was correctly estimated. Our approach, based on probabilistic inference concerning decision-making, can lead to further improvement in robot autonomy.

A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots

“…By maximizing the trajectory probability based on the CaI framework, path planning based on semantic information can be conducted using the PGM, that is, SpCoSLAM. This method is called SpCoNavi (Taniguchi et al, 2020a). These results clearly demonstrate that the PGM-based approach has the flexibility of integrating a broad range of cognitive modules and the capability to make them learn together.…”

Map completion from partial observation using the global structure of multiple environmental maps

“…The occupancy grid map is one of the map representations that robots commonly use to accomplish various service tasks [1,[15][16][17]. It is a method that divides the environment into grid cells at fixed intervals and stores the occupancy probability of each cell in a 2D list format.…”

Hierarchical Bayesian Model for the Transfer of Knowledge on Spatial Concepts based on Multimodal Information

“…This model has no mechanisms to transfer inferred parameters between environments. They proposed SpCoNavi [48], which estimates the route to the target place based on human linguistic instructions such as 'Go to kitchen' by utilizing the spatial concept learned in Online SpCoSLAM [49]. Isobe et al proposed a spatial conceptual model that utilized object information as the bag of objects obtained from YOLO9000 [50] instead of the image features [51].…”