Off-line simulation inspires insight: A neurodynamics approach to efficient robot task learning

Abstract

There is currently an increasing demand for robots able to acquire the sequential organization of tasks from social learning interactions with ordinary people. Interactive learning-by-demonstration and communication is a promising research topic in current robotics research. However, the efficient acquisition of generalized task representations that allow the robot to adapt to different users and contexts is a major challenge. In this paper, we present a dynamic neural field (DNF) model that is inspired by the hypothesis that the nervous system uses the off-line re-activation of initial memory traces to incrementally incorporate new information into structured knowledge. To achieve this, the model combines fast activation-based learning to robustly represent sequential information from single task demonstrations with slower, weight-based learning during internal simulations to establish longer-term associations between neural populations representing individual subtasks. The efficiency of the learning process is tested in an assembly paradigm in which the humanoid robot ARoS learns to construct a toy vehicle from its parts. User demonstrations with different serial orders together with the correction of initial prediction errors allow the robot to acquire generalized task knowledge about possible serial orders and the longer term dependencies between subgoals in very few social learning interactions. This success is shown in a joint action scenario in which ARoS uses the newly acquired assembly plan to construct the toy together with a human partner.

Developing Dynamic Field Theory Architectures for Embodied Cognitive Systems with cedar

“…Beyond the toy example used here, DFT architectures have exploited the scaling properties of DFT to push both toward generating motor behaviors in autonomous robots (Knips et al, 2014; Strauss et al, 2015; Zibner et al, 2015) and toward higher cognitive function, such as grounding spatial language (Richter et al, 2014a), parsing action sequences (Lobato et al, 2015), or task learning (Sousa et al, 2015). These architectures are fairly complex.…”

A neural integrator model for planning and value-based decision making of a robotics assistant

“…The first experiment is a pipe assembly task in which the robot learns the sequential order of handing over different pipes to an operator who performs the assembly steps. We apply a learning by demonstration paradigm which has been successfully used in the past to teach robots sequential tasks [9,21,55]. An important practical prerequisite is that robot learning should be efficient and fast since the human tutors cannot be expected to repeat the demonstration of the action sequence several times [45].…”

“…Since the robot is not directly involved in the assembly work, only the sequence of pipe transfers between the assistant and the operator is demonstrated for simplicity. Note however that the neural computations support fast serial order learning of the entire assembly sequence including the manipulation of pipes initially located in the operator's workspace [55]. The learning is guided by the information provided by the vision system about the length and the hue value in color space coordinates of each pipe.…”

Combining intention and emotional state inference in a dynamic neural field architecture for human-robot joint action

“…It is assumed that both partners know the construction plan and keep track of the subtasks that have been already completed by the team. The prior knowledge about the sequential execution of the assembly work is represented in layer CSGL of the DNF-architecture, by connections between populations encoding subsequent assembly steps (for how these connections could have been established through learning by demonstration and tutor's feedback see Sousa et al (2015)). Since the desired end state does not uniquely define the logical order of the construction, at each stage of the construction the execution of several subtasks may be simultaneously possible.…”

“…In line with this finding, CSGL contains two connected DNF layers with population representations of past and future events. The connections linking the neural populations in one DNF to the other DNF encode the different serial order of subgoals of the task (see Sousa, Erlhagen, Ferreira, and Bicho (2015) for how these can be learned by tutor demonstration and feedback).…”