Structural descriptions in human-assisted robot visual learning

Abstract

The paper presents an approach to using structural descriptions, obtained through a human-robot tutoring dialogue, as labels for the visual object models a robot learns. The paper shows how structural descriptions enable relating models for different aspects of one and the same object, and how being able to relate descriptions for visual models and discourse referents enables incremental updating of model descriptions through dialogue (either robot-or human-initiated). The approach has been implemented in an integrated architecture for human-assisted robot visual learning.

Incremental generation of spatial referring expressions in situated dialog

“…We have employed the approach in a system for Human-Robot Interaction, in the setting of object manipulation in natural scenes. For more detail, see (Kruijff et al, 2006a;Kruijff et al, 2006b).…”

Referring to the recently seen: reference and perceptual memory in situated dialog

“…For example, a pronominal reference signals that the intended referent has a high degree of salience within the hearer's current mental model of the discourse context. 1 Examples of such research spanning the decades include [58,38,45,32,20,39,48,13,64,63,34,26,15,62,52] 2 Herskovits [27] provides an excellent overview of the challenges posed by spatial language. Many computational models of spatial language are based on the spatial template concept proposed by Logan and Sadler [54]; see [19,40,10,37] for examples of spatial template based computational models of the semantics of topological prepositions, and [18,42,5] for computational models of projective prepositions.…”

Robot Controllers for Highly Dynamic Environments with Real-time Constraints

“…The project comprises of work on architectures, environment representation, object recognition, human-machine interaction and natural language understanding, planning, or action representations (see e.g. Roth et al 2005;Meier et al 2006;Kruijff et al 2006)). One of the key ideas of this project is to integrate work on the different fields of AI and robotics research as well as the cognitive sciences.…”