The distribution of rewards in sensorimotor maps acquired by cognitive robots through exploration

Mohan, Vishwanathan; Morasso, Pietro; Metta, Giorgio; Kasderidis, Stathis

doi:10.1016/j.neucom.2011.06.009

Cited by 8 publications

(11 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mohan, Morasso and Metta [22] built on Toussaint sensorimotor map [18] in developing a mechanism that allows a GNOSYS robot to learn 'when' to optimize 'what constraint' while realising a spatial goal and to be able to push a ball intelligently to the corners of a table while avoiding traps in arbitrary positions on the table. A key contribution of the paper is in the design of a bifurcation parameter, which measures the anticipatory ability of the model.…”

Section: Review Of Related Workmentioning

confidence: 99%

Unsupervised Temporospatial Neural Architecture for Sensorimotor Map Learning

Ezenkwu

Starkey

2021

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

The ability to learn the sensorimotor maps of unknown environments without supervision is a vital capability of any autonomous agent, be it biological or artificial. An accurate sensorimotor map should be able to encode the agent's world and equip it with the capability to anticipate or predict the results of its actions. However, to design a robust autonomous learning technique for an unknown, dynamic, partially observable or noisy environment remains a daunting task. This paper proposes a Temporospatial Merge Grow When Required (TMGWR) network for continuous self-organisation of an agent's sensorimotor awareness in noisy environments. TMGWR is an adaptive neural algorithm that learns the sensorimotor map of an agent's world using a time series self-organising strategy and the Grow When Required (GWR) algorithm. The algorithm is compared with GNG, GWR and TGNG in terms of their disambiguation performance, sensorial representation accuracy and sensorimotor-link error, a new metric that is developed in this paper to evaluate how well a sensorimotor map represents causality in the agent's world. The outcomes of the experiments show that TMGWR is more efficient and suitable for sensorimotor map learning in noisy environments than the competing algorithms.

show abstract

Section: Review Of Related Workmentioning

confidence: 99%

Unsupervised Temporospatial Neural Architecture for Sensorimotor Map Learning

Ezenkwu

Starkey

2021

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…2009;Hesslow 2012). The proposed neural framework further exploits several interesting computational conce pts in literature mainly, (a) the idea of growing neural gas (Fritzke 1995) which is an extension of selforganizing maps (Kohonen et al 2001) for internal representation of the peripersonal space; (b) Ne ural field dynamics (Amari 1977;Erlhagen & Schöner 2002) to organize goal directed reasoning and cooperative behavior in shared spaces; (c) the idea of reward fields (Toussaint 2006) for reasoning about action sequences based on the unfolding spatio-temporal dynamics of the task (d) learning of such behavior-modulating reward fields through experience gained by exploration (Mohan, Morasso, Metta, et al 2011) (e) the idea of Passive Motion Paradigm as an internal representation of the bodi e s of the agents for simulation, reasoning and execution of goal directed actions. The architecture presented here further builds up on (Toussaint 2006;Mohan, Morasso, Metta, et al 2011) by hypothesizing the synergistic interaction between two coupled internal models for reasoning and control, employing the architecture on multiple embodiments to reasoning and goal oriented cooperation in a real world industrial assembly line.…”

Section: A Case For Coupled Memory Representations Of Body and Space mentioning

confidence: 99%

“…The proposed neural framework further exploits several interesting computational conce pts in literature mainly, (a) the idea of growing neural gas (Fritzke 1995) which is an extension of selforganizing maps (Kohonen et al 2001) for internal representation of the peripersonal space; (b) Ne ural field dynamics (Amari 1977;Erlhagen & Schöner 2002) to organize goal directed reasoning and cooperative behavior in shared spaces; (c) the idea of reward fields (Toussaint 2006) for reasoning about action sequences based on the unfolding spatio-temporal dynamics of the task (d) learning of such behavior-modulating reward fields through experience gained by exploration (Mohan, Morasso, Metta, et al 2011) (e) the idea of Passive Motion Paradigm as an internal representation of the bodi e s of the agents for simulation, reasoning and execution of goal directed actions. The architecture presented here further builds up on (Toussaint 2006;Mohan, Morasso, Metta, et al 2011) by hypothesizing the synergistic interaction between two coupled internal models for reasoning and control, employing the architecture on multiple embodiments to reasoning and goal oriented cooperation in a real world industrial assembly line. Figure 12: A-D) Show the typically different possible setups in which objects can be lying in the workspace; A) shows a structured scenario where a fuse box with 3 fuses is near each robot and parallel assembly c a n be ea s i l y realized; B) An unstructured setup where objects are lying in random places and reasoning is essential for successful joint assembly; C) An unstructured and redundant setup where there are more than two fuse boxes to work with; D) A scenario where none of the two robots can reach all the desired objects, so the assembly is not realizable in a direct way without mutual cooperation and spatial reasoning; E) shows the robotic s platform used for experiments with two robots, the camera and the workspace tray.…”

Section: A Case For Coupled Memory Representations Of Body and Space mentioning

confidence: 99%

Towards a Brain-like Memory for Cognitive Robots. PhD Dissertation, University of Genova, Genova, Italy

Bhat¹

2018

Preprint

View full text Add to dashboard Cite

The most striking feature of any cognitive system is its ability to keep learning cumulatively and exploit its experiences prospectively to anticipate, reason and produce goal-oriented behaviors. At the core of this intriguing capability lies our memory. Every touch, sound, sight, taste triggers our memory, enabling us to remember and flexibly connect our past experiences, with the available present to the possible future. While emerging trends in neurosciences are rapidly enriching our understanding of the functional organization of memory in the brain, a prospective memory architecture is also a central design feature necessary if robots are to truly become commonplace assistants in numerous application domains (domestic, industrial, others) and in the complexity of the environments we inhabit and create (where neither everything can be known nor can everything be experienced). Hence, with the central premise that cognition is constructive manipulation of memory, this thesis proposes a novel brain guided perspective on the design of cognitive architectures for cumulatively developing systems. Set up in the context of several experiments from animal and infant cognition reenacted on the iCub humanoid, a principled framework for cumulative learning of actions, skills, affordances and cause-effect relations through multiple streams (imitation, exploration, linguistic inputs) is proposed. How such diverse experiences of the robot learnt in past are recalled based on present context, combined with explorative actions to learn something new, creatively connected to generate novel behaviors in the context of sought goals is demonstrated through several tasks. The proposed integrated machinery provides for a shared computational basis for recalling of the past and simulation of the future in line with the emerging trends from neuroscience (like the brain’s default mode network). A notable transition further is the porting of the framework developed on iCub humanoid to real world industrial settings in tasks like assembly and manufacturing (facilitating runtime reasoning and quick switchover to novel assembly tasks). In this sense, the proposed architecture brings together the fields of Robotics, Developmental Psychology, and Neuroscience to craft cognitive and adaptable End User Applications while at the same time provides insights into the functional organization of the brain. In parallel, it lays the foundation for a new generation of cognitive architectures that are domain agnostic (i.e. support open-ended learning), partially embodiment agnostic (i.e. are configurable to new robotic platforms), partially self-driven (i.e. can generate their own internal goals) and brain guided.

show abstract

“…Chimps can easily solve this problem of combinatorial tool use (Maravita and Iriki, 2004) and since the observation of their behavior rules out the possibility of trial-and-error, the most likely interpretation is as follows: (1) the chimp has an abstract concept of a stick-like object which must have similar computational properties to the body schema in order to be integrated with it, at least temporarily in the course of the task (Iriki et al, 1996); (2) the recognition of crucial “affordances,” such as the fact that the food and the long stick are unreachable and the short stick is reachable and long enough to get the long stick, is carried out by means of covert, “imagined” movements. In a previous work (Mohan and Morasso, 2007, 2011d) we have shown how adding a reasoning system on top of the PMP-based real/mental action generation system can enable a cognitive robot (GNOSYS) to autonomously generate goal directed plans in such scenarios (where use of tools is obligatory for achieving the goal). Figure 8 illustrates the sequences of (real and virtual) actions initiated by iCub using different task-specific PMP networks (illustrated in different examples so far) when it exploits a long green stick as a tool to reach (an otherwise unreachable) red cylinder.…”

Section: Oct and Pmp As Computational Theoriesmentioning

confidence: 99%

Passive Motion Paradigm: An Alternative to Optimal Control

Mohan¹,

Morasso²

2011

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

In the last years, optimal control theory (OCT) has emerged as the leading approach for investigating neural control of movement and motor cognition for two complementary research lines: behavioral neuroscience and humanoid robotics. In both cases, there are general problems that need to be addressed, such as the “degrees of freedom (DoFs) problem,” the common core of production, observation, reasoning, and learning of “actions.” OCT, directly derived from engineering design techniques of control systems quantifies task goals as “cost functions” and uses the sophisticated formal tools of optimal control to obtain desired behavior (and predictions). We propose an alternative “softer” approach passive motion paradigm (PMP) that we believe is closer to the biomechanics and cybernetics of action. The basic idea is that actions (overt as well as covert) are the consequences of an internal simulation process that “animates” the body schema with the attractor dynamics of force fields induced by the goal and task-specific constraints. This internal simulation offers the brain a way to dynamically link motor redundancy with task-oriented constraints “at runtime,” hence solving the “DoFs problem” without explicit kinematic inversion and cost function computation. We argue that the function of such computational machinery is not only restricted to shaping motor output during action execution but also to provide the self with information on the feasibility, consequence, understanding and meaning of “potential actions.” In this sense, taking into account recent developments in neuroscience (motor imagery, simulation theory of covert actions, mirror neuron system) and in embodied robotics, PMP offers a novel framework for understanding motor cognition that goes beyond the engineering control paradigm provided by OCT. Therefore, the paper is at the same time a review of the PMP rationale, as a computational theory, and a perspective presentation of how to develop it for designing better cognitive architectures.

show abstract

The distribution of rewards in sensorimotor maps acquired by cognitive robots through exploration

Cited by 8 publications

References 38 publications

Unsupervised Temporospatial Neural Architecture for Sensorimotor Map Learning

Unsupervised Temporospatial Neural Architecture for Sensorimotor Map Learning

Towards a Brain-like Memory for Cognitive Robots. PhD Dissertation, University of Genova, Genova, Italy

Passive Motion Paradigm: An Alternative to Optimal Control

Contact Info

Product

Resources

About