Interpretation of Spatial Language in a Map Navigation Task

Abstract

Abstract-We have developed components of an automated system that understands and follows navigational instructions. The system has prior knowledge of the geometry and landmarks of specific maps. This knowledge is exploited to infer complex paths through maps based on natural language descriptions. The approach is based on an analysis of verbal commands in terms of elementary semantic units that are composed to generate a probability distribution over possible spatial paths in a map. An integration mechanism based on dynamic programming guides this language-to-path translation process, ensuring that resulting paths satisfy continuity and smoothness criteria. In the current implementation, parsing of text into semantic units is performed manually. Composition and interpretation of semantic units into spatial paths is performed automatically. In the evaluations, we show that the system accurately predicts the speakers' intended meanings for a range of instructions. This paper provides building blocks for a complete system that, when combined with robust parsing technologies, could lead to a fully automatic spatial language interpretation system. Index Terms-Human-machine interaction, natural language processing, navigational instructions, spatial language understanding.

Toward understanding natural language directions

“…Our work follows their methodology of corpus-based robotics but focuses on achieving robust understanding of natural language directions rather than an end-to-end system. Levit and Roy [2] designed navigational informational units that break down instructions into components. MacMahon et al [3] represented a clause in a set of directions as a compound action consisting of a simple action (move, turn, verify, and declaregoal), plus a set of pre-and post-conditions.…”

“…Look et al [4] created an ontology for modeling the relationships between spaces, and used it for generating natural language directions. Many of these representations are more expressive than SDCs but are more difficult to automatically extract from text; many authors sidestep this problem by using human annotations (e.g., [2,3]). …”

Natural language command of an autonomous micro-air vehicle

“…The notion of spatial description clauses is influenced by many similar formalisms for reasoning about the semantics of natural language directions [7], [8], [9], [10], [11], [12]. The structure of the spatial description clause builds on the work of Landau and Jackendoff [13], and Talmy [14], providing a computational instantiation of their formalisms.…”

Multimodal cognitive interface for robot navigation

“…They defined a lexicon of words in terms of spatial routines and used that lexicon to build a speech-controlled robot in a simulator. Levit and Roy (2007) have developed components of an automated system that understands and follows navigational instructions. The system has prior knowledge of the geometry and landmarks of specific maps.…”