Learning where to look with movement-based intrinsic motivations: A bio-inspired model

Abstract

Most sophisticated mammals, in particular primates, interact with the world to acquire knowledge and skills later exploitable to obtain biologically relevant resources. These interactions are driven by intrinsic motivations. Recent research on brain is revealing the system of neural structures, pivoting on superior colliculus, underlying trial-and-error learning processes guided by movement-detection, one important element of one specific type of intrinsic motivation mechanism. Here we present a preliminary computational model of such system guiding the acquisition of overt attentional skills. The model is formed by bottom-up attentional components, exploiting the intrinsic properties of the scene, and top-down attentional components, learning under the guidance of movement-based intrinsic motivation. The model is tested with a simple task, inspired by the 'gazecontingency paradigm' proposed in cognitive psychology, where looking some portions of the environment can directly change it. The tests of the model show how its integrated components can learn skills causing relevant changes in the environment while ignoring changes non-contingent to own action. The model also allows the presentation of a wider research agenda directed to build biologically plausible models of the interaction between overt attention control and intrinsic motivations.

An Embodied Agent Learning Affordances With Intrinsic Motivations and Solving Extrinsic Tasks With Attention and One-Step Planning

“…A last critical component is the simple bottom-up attention mechanism used to identify objects, and, as expected, this was limited (it scaled worse than linearly with the number of objects). The component could be enhanced with the addition of more sophisticated top-down attention mechanisms able to drive attention on the basis of the current knowledge on the identity and position of objects in the scene (Rasolzadeh et al, 2010; Sperati and Baldassarre, 2014, 2018; Ognibene and Baldassare, 2015).…”

“…The process works as follows. Firstly the system focuses on the portion of space where a change in the periphery image takes place (this mimics some processes of primates for which a reflex focuses attention on changes happening in the environment (Comoli et al, 2003; Gandhi and Katnani, 2011; Sperati and Baldassarre, 2014). To this purpose, the system computes the “change image” given by the pixel-by-pixel absolute difference between the whole periphery image after and before the performance of the action: both images are taken with the same initial gaze before the next attentional movement but the “after-image” is taken after the action performance (e.g., leading to displace the object).…”

“…The system was developed to accomplish only one extrinsic task, rather than multiple ones as here, and did not deal with open-ended learning. A system developed starting from the previous ones (Sperati and Baldassarre, 2014), again endowed with a bottom-up and a top-down attention component, is instead able to self-generate and learn tasks based on intrinsic motivations. This system has been further developed to self-generate and encode multiple visual-target goals , and to learn the camera-controller to find them (Sperati and Baldassarre, 2018).…”

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

“…In this respect, the amount of competition between competing bottom-up information sources, possibly based on parietal cortex (which represents an important source of input to superior colliculus and the basal ganglia areas controlling eye movements, [46]) has been suggested to play an important role in the development of attention [92]. In this respect, additional efforts will be spent to constrain the model architecture and functioning with general neuroscietific knowledge [28], [94] and neuroscientific knowledge related to development [55]. f) Motor system -future developments and new challenges for attention: The possibilities of enhancing the motor components of the system are several as motor control involves a whole set of challenges on its own, for example to implement reaching, grasping, obstacle avoidance, sophisticated movement trajectories, cyclic movements, multiple movements to solve different tasks [8].…”