Framework and implications of virtual neurorobotics

Abstract

Despite decades of societal investment in artifi cial learning systems, truly "intelligent" systems have yet to be realized. These traditional models are based on input-output pattern optimization and/or cognitive production rule modeling. One response has been social robotics, using the interaction of human and robot to capture important cognitive dynamics such as cooperation and emotion; to date, these systems still incorporate traditional learning algorithms. More recently, investigators are focusing on the core assumptions of the brain "algorithm" itself-trying to replicate uniquely "neuromorphic" dynamics such as action potential spiking and synaptic learning. Only now are large-scale neuromorphic models becoming feasible, due to the availability of powerful supercomputers and an expanding supply of parameters derived from research into the brain's interdependent electrophysiological, metabolomic and genomic networks. Personal computer technology has also led to the acceptance of computer-generated humanoid images, or "avatars", to represent intelligent actors in virtual realities. In a recent paper, we proposed a method of virtual neurorobotics (VNR) in which the approaches above (social-emotional robotics, neuromorphic brain architectures, and virtual reality projection) are hybridized to rapidly forward-engineer and develop increasingly complex, intrinsically intelligent systems. In this paper, we synthesize our research and related work in the fi eld and provide a framework for VNR, with wider implications for research and practical applications.

Brainlab: a Python toolkit to aid in the design, simulation, and analysis of spiking neural networks with the NeoCortical Simulator

“… A convenient environment in which to convert a neural network description into a chromosomal representation suitable for use with a genetic algorithm. A convenient environment in which to access NCS's realtime stimulus input capabilities, especially for robotic interface applications (see Goodman et al, 2008 for more information on using NCS in robotics). The ability to conveniently extend many of these capabilities without recourse to coding in NCS's native compiled programming environment (the C/C++ language).…”

“…A convenient environment in which to access NCS's realtime stimulus input capabilities, especially for robotic interface applications (see Goodman et al, 2008 for more information on using NCS in robotics).…”

Virtual Partner Interaction (VPI): Exploring Novel Behaviors via Coordination Dynamics

“…for perception, attention, spatial navigation, learning, decision-making and so forth. The second, guided by principles of phylogeny and ontogeny, attempts to self-organize basic building blocks into purposeful systems ( [33] , [40] – [42] ; see also [43] for a similar goal at the neural level). Ever greater recognition is being given to the importance of coordination between the agent's “brain” and “body” [27] , [33] , [34] , [41] , [44] , as well as to the social significance of behavior [33] , [44] , [45] .…”

Reward-based learning for virtual neurorobotics through emotional speech processing

“…As described by our previous studies by Goodman et al (2007, 2008), we define VNR as follows: a computer-facilitated behavioral loop wherein a human interacts with a projected robot that meets five criteria: the robot is sufficiently embodied for the human to tentatively accept the robot as a social partner; the loop operates in real time, with no pre-specified parcellation into receptive and responsive time windows; the cognitive control is a neuromorphic brain emulation using our NeoCortical simulator (NCS) and incorporating realistic neuronal dynamics whose time constants reflect synaptic activation, membrane and circuitry properties, and most importantly learning; the neuromorphic architecture is expandable to progressively larger scale and complexity to track brain development; and the neuromorphic architecture can potentially provide circuitry underlying intrinsic motivation and intentionality, which physiologically is best described as emotional rather than rule-based drive.…”

“…For the past couple of years, we have worked on machine learning systems, and we developed a Virtual Neurorobotic (VNR) loop, which focuses on the coupling of neural systems with some form of physical actuation. This is based around the interoperability of a neural model, a virtual robotic avatar and a human participant (Goodman et al, 2007, 2008). Under all but the most basic scenarios this interoperability is accomplished through an organized network communication system (Thibeault et al, 2010b, 2012).…”