Robot Cognitive Control with a Neurophysiologically Inspired Reinforcement Learning Model

Abstract

A major challenge in modern robotics is to liberate robots from controlled industrial settings, and allow them to interact with humans and changing environments in the real-world. The current research attempts to determine if a neurophysiologically motivated model of cortical function in the primate can help to address this challenge. Primates are endowed with cognitive systems that allow them to maximize the feedback from their environment by learning the values of actions in diverse situations and by adjusting their behavioral parameters (i.e., cognitive control) to accommodate unexpected events. In such contexts uncertainty can arise from at least two distinct sources – expected uncertainty resulting from noise during sensory-motor interaction in a known context, and unexpected uncertainty resulting from the changing probabilistic structure of the environment. However, it is not clear how neurophysiological mechanisms of reinforcement learning and cognitive control integrate in the brain to produce efficient behavior. Based on primate neuroanatomy and neurophysiology, we propose a novel computational model for the interaction between lateral prefrontal and anterior cingulate cortex reconciling previous models dedicated to these two functions. We deployed the model in two robots and demonstrate that, based on adaptive regulation of a meta-parameter β that controls the exploration rate, the model can robustly deal with the two kinds of uncertainties in the real-world. In addition the model could reproduce monkey behavioral performance and neurophysiological data in two problem-solving tasks. A last experiment extends this to human–robot interaction with the iCub humanoid, and novel sources of uncertainty corresponding to “cheating” by the human. The combined results provide concrete evidence for the ability of neurophysiologically inspired cognitive systems to control advanced robots in the real-world.

Artificial cognitive control with self-x capabilities: A case study of a micro-manufacturing process

“…Nevertheless, reinforcement learning has been explored in artificial cognitive control by means of computational models to control robotic systems [5]. Recent studies corroborate what has been known for a long time: automatic and flexible decision-making procedures are the cornerstone to reduce human intervention in the presence of complexity, uncertainty, background noise, and large data volumes typical of production systems [6,7].…”

Learning Robotic Hand-eye Coordination Through a Developmental Constraint Driven Approach

“…Khamassi et al [12] inspired us that we can build a computational model to simulate part of brain working loop. However, they only focus on building a complex system, rather than let a system develop to be complex.…”

“…The nonlinear approximation ability of the artificial neural network supports the implementations of the hand-eye mapping very well. In particular, [10,12] used radial basis function (RBF) networks to simulate V6A cortex in the human brain. Self-organizing map networks and Jacobian matrices are able to deal with the kinematic redundancy of the robot [11,29] .…”

An Infant Development-inspired Approach to Robot Hand-eye Coordination

“…Also, several recent works consider a simulated human brain structure to solve the problem of hand-eye mapping. For example, [12,13] used radial basis function networks to simulate the V6A cortex in the human brain. [14, 22-24, 18, 34] applied neural networks to mimic the brain's distinct cortices.…”

“…In studies [10,11], a developmental learning algorithm was applied to obtain this type of transformation. Other developmental robotic hand-eye coordination systems [7,[12][13][14][19][20][21] have used different neural networks to simulate some of the brain loops that control hand-eye coordination. Such research indicates that introducing brain-like structures into developmental robotics is regarded as an effective solution to robotic cognition [2].…”