Self-Imitation and Environmental Scaffolding for Robot Teaching

Abstract

Imitative learning and learning by observation are social mechanisms that allow a robot to acquire knowledge from a human or another robot. However to be able to obtain skills in this way the robot faces many complex issues, one of which is that of finding solutions to the correspondence problem. Evolutionary predecessors to observational imitation may have been self-imitation where an agent avoids the complexities of the correspondence problem by learning and replicating actions it has experienced through the manipulation of its body. We investigate how a robotic control and teaching system using self-imitation can be constructed with reference to psychological models of motor control and ideas from social scaffolding seen in animals. Within these scaffolded environments sets of competencies can be built by constructing hierarchical state/action memory maps of the robot's interaction within that environment. The scaffolding process provides a mechanism to enable learning to be scaled up. The resulting system allows a human trainer to teach a robot new skills and modify skills that the robot may possess. Additionally the system allows the robot to notify the trainer when it is being taught skills it already has in its repertoire and to direct and focus its attention and sensor resources to relevant parts of the skill being executed. We argue that these mechanisms may be a first step towards the transformation from self-imitation to observational imitation. The system is validated on a physical pioneer robot that is taught using self-imitation to track, follow and point to a patterned object.

Helping robots imitate: metrics and technological solutions inspired by human behaviour

“…Ideas of robot feedback are also explored by Saunders et al [24]. In this instance the robot is effectively aware of what it already 'knows' in the form of a hierarchy of behavioural competencies.…”

“…In terms of spontaneous segmentation of tasks, we described in [7] how, in a robotic experiment (using the architecture described in [24]), that segmenting a behaviour not only allows task re-use and reduces human training time but can also serve to enhance the algorithmic effect in the environmental scaffolding process (in this case information gain). Thus the robot benefits with better performance, both on the task taught and from reduced computation.…”

“…A partial technological solution to these issues has been addressed in [24] which describes a novel approach inspired by the notion of "zone of proximal development" initially proposed by Vygotsky [25,26], and demonstrates how teaching a robot can be achieved through active manipulation of the robot's actuators. Behavioural competencies are built step-by-step by the human, exploiting those competencies already taught.…”

An integrated three-stage model towards grammar acquisition

“…A perceptioncapable robot can decide what a sound unit that it considers a word may stand for and try out uttering it. Our hypothesis is that, following our previous work on social learning of behaviour acquisition [26], the adult utterances situated in the learning context, together with reinforcement via forms of affective feedback and communicative success or failure (e.g. in shared intentional reference), allow enough bias for fast learning to take place.…”

Issues in Human/Robot Task Structuring and Teaching

“…In our previous work we have developed, implemented and validated a robot social learning architecture on various physical robotic platforms [1]. The architecture allows a possibly naive human teacher to teach a robot skills and competencies in a flexible, extendable and open-ended way.…”