Learning environmental knowledge from task-based human-robot dialog

Abstract

This paper presents an approach for learning environmental knowledge from task-based human-robot dialog. Previous approaches to dialog use domain knowledge to constrain the types of language people are likely to use. In contrast, by introducing a joint probabilistic model over speech, the resulting semantic parse and the mapping from each element of the parse to a physical entity in the building (e.g., grounding), our approach is flexible to the ways that untrained people interact with robots, is robust to speech to text errors and is able to learn referring expressions for physical locations in a map (e.g., to create a semantic map). Our approach has been evaluated by having untrained people interact with a service robot. Starting with an empty semantic map, our approach is able ask 50% fewer questions than a baseline approach, thereby enabling more effective and intuitive human robot dialog.

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

“…In [25], background knowledge about robot actions is axiomatized using Markov Logic Networks. In [26], a knowledge base of known actions, objects, and locations is used for a Bayesbased grounding model. Symbolic approaches work well for small pre-defined domains, but most of them employ manually written rules, which limits their coverage and scalability.…”

Multi-robot task acquisition through sparse coordination

“…We refer the reader to the original work on Instruction Graphs for a more detailed overview [15]. commands are processed by a probabilistic parser and grounder [20]. This allows the robot to learn the groundings from natural language to robot primitives, environmental features, and tasks.…”

“…Instead, Figures 5a and 5b show the corresponding Sparse Coordination Instruction Graphs. In this specific case, the natural language description of the tasks was parsed through the aid of specifically developed parsers, similarly to a previous work [20]. The descriptions were grounded to objects and locations of a knowledge base containing a high-level description of the environment and the robot primitives.…”

BWIBots: A platform for bridging the gap between AI and human–robot interaction research

“…At the intersection of dialog and language grounding, past work presented a dialog agent used together with a knowledge base and understanding component to learn new referring expressions during conversations that instruct a mobile robot (Kollar, Perera, Nardi and Veloso 2013). They use semantic frames of actions and arguments extracted from user utterances, while we use λ-calculus meaning representations.…”