Grounding spatial relations for human-robot interaction

Natural Language Communication with Robots

“…Progress on language in this area has largely focused on grounding visual attributes (Kollar et al, 2013;Matuszek et al, 2014) and on learning spatial relations and actions for small vocabularies with hard-coded abstract concepts (Steels and Vogt, 1997;Roy, 2002;Guadarrama et al, 2013). Language is sometimes grounded into simple actions (MacMahon et al, 2006;Yu and Siskind, 2013) but the data, while multimodal, is relatively formulaic, the vocabularies are small, and the grammar is constrained.…”

Situated Interaction with a Smart Environment: Challenges and Opportunities

“…The main prerequisite for this kind of strategy is to understand the semantic contribution made by each of the spatial prepositions (or preposition-like structures, such as the phrase in the front of). This can be treated either independently of the contextual details conveyed by experiential knowledge of objects and their features and functional interrelationships, or by identifying generic functional features that can be implemented in an adaptive system [15]. In this regard, different types of prepositions pose different types of challenges, which can be briefly sketched as follows (see Tenbrink [38] for further details of spatial term categories).…”

“…While there have been many efforts to capture the notion of common ground in general [8] and in human-robot interaction settings [6], the computational management of situated spatial dialogue is still under-developed [15,34] and requires creative solutions for reference handling [23], including attempts to incorporate the human's gaze in the system's interpretation procedure [1], and strategies for handling errors [33]. One major challenge concerns the fundamental difference between human concepts represented by natural language, especially in the domain of space [2], and formal systems suited for computational purposes, e.g., spatial reasoning-even if based on qualitative rather than metric relations [25].…”

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions

“…Some recent notable works have explored aspects of this problem (e.g., [33,45,7,4,18]). Guadarrama et al [18] focused on using spatial relations to ground NL objects to objects in the environment.…”

“…Guadarrama et al [18] focused on using spatial relations to ground NL objects to objects in the environment. They used an injective mapping from verbs to controller instructions based on pre-defined templates.…”