Spatial Language for Human–Robot Dialogs

Abstract

Report Documentation PageForm Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. The original document contains color images. REPORT DATE ABSTRACTIn conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users. In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog. Examples using spatial language are included for both robot-to-human feedback and also human-to-robot commands. We also discuss some linguistic consequences in the semantic representations of spatial and locative information based on this work. AbstractIn conversation, people often use spatial relationships to describe their environment, e.g., "There is a desk in front of me and a doorway behind it", and to issue directives, e.g., "Go around the desk and through the doorway." In our research, we have been investigating the use of spatial relationships to establish a natural communication mechanism between people and robots, in particular, for novice users.In this paper, the work on robot spatial relationships is combined with a multi-modal robot interface. We show how linguistic spatial descriptions and other spatial information can be extracted from an evidence grid map and how this information can be used in a natural, human-robot dialog. Examples using spatial language are included for both robot-to-human feedback and also human-to-robot commands. We also discuss some linguistic consequences in the semantic representations of spatial and locative information based on this work.

Natural language command of an autonomous micro-air vehicle

“…The notion of spatial description clauses is influenced by many similar formalisms for reasoning about the semantics of natural language directions [7], [8], [9], [10], [11], [12]. The structure of the spatial description clause builds on the work of Landau and Jackendoff [13], and Talmy [14], providing a computational instantiation of their formalisms.…”

Using Spatial Semantic and Pragmatic Fields to Interpret Natural Language Pick-and-Place Instructions for a Mobile Service Robot

“…Previous work that has investigated the use and representation of spatial language in human-robot interaction (HRI) includes the work of Skubic et al [9], who demonstrated a robot capable of understanding and relaying static spatial relations in natural language instruction and production tasks. Computational field models of static relations have also been used in systems for object pick-and-place tasks on a tabletop [8], and for visually situated dialogue [11].…”

A Preliminary Study of Peer-to-Peer Human-Robot Interaction

“…When human-robot teams perform operational tasks, such as construction, understanding and communicating spatial dialogue plays a significant role [8]- [10]. In particular, when humans and robots work in a shared environment, robots must be able to understand how humans perceives space and the relative positions of objects around them.…”