Event-driven visual attention for the humanoid robot iCub

Rea, Francesco; Metta, Giorgio; Bartolozzi, Chiara

doi:10.3389/fnins.2013.00234

Cited by 35 publications

(30 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On one side, this debate involves positions that, also based on psychological evidence (e.g., in relation to pop-out effects, [107]), propose models that assign a prominent role to bottom-up features guiding visual exploration on the basis of stimulus-based saliency maps [53], [58], [88]. On the other side, the debate sees positions stressing the pivotal role of top-down task-dependent processes to explain human visual attention [41], [65], [79], [90], [96], [102].…”

Section: The Bottom-up Componentmentioning

confidence: 99%

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

Ognibene

Baldassare²

2015

IEEE Trans. Auton. Mental Dev.

View full text Add to dashboard Cite

Vision gives primates a wealth of information useful to manipulate the environment, but at the same time it can easily overwhelm their computational resources. Active vision is a key solution found by nature to solve this problem: a limited fovea actively displaced in space to collect only relevant information.Here we highlight that in ecological conditions this solution encounters four problems: (1) the agent needs to learn where to look based on its goals; (2) manipulation causes learning feedback in areas of space possibly outside the attention focus; (3) good visual actions are needed to guide manipulation actions, but only these can generate learning feedback; (4) a limited fovea causes aliasing problems. We then propose a computational architecture ('BITPIC') to overcome the four problems, integrating four bioinspired key ingredients: (1) reinforcement-learning fovea-based top-down attention; (2) a strong vision-manipulation coupling; (3) bottom-up periphery-based attention; (4) a novel action-oriented memory. The system is tested with a simple simulated cameraarm robot solving a class of search-and-reach tasks involving colour-blob 'objects'. The results show that the architecture solves the problems, and hence the tasks, very efficiently, and highlight how the architecture principles can contribute to a full exploitation of the advantages of active vision in ecological conditions.

show abstract

Section: The Bottom-up Componentmentioning

confidence: 99%

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

Ognibene

Baldassare²

2015

IEEE Trans. Auton. Mental Dev.

View full text Add to dashboard Cite

show abstract

“…The latency of an event-based visual attention was two order less than frame-based one (Rea et al, 2013). Recognition of playing-card suit was achieved as a deck was flicked through (30 ms exposure) (Serrano-Gotarredona and Linares-Barranco, 2015).…”

Section: Event-driven Vision For Robotsmentioning

confidence: 99%

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

et al. 2018

Self Cite

View full text Add to dashboard Cite

Citation:Glover A, Vasco V, Iacono M and Bartolozzi C (2018) Event-driven (ED) cameras are an emerging technology that sample the visual signal based on changes in the signal magnitude, rather than at a fixed-rate over time. The change in paradigm results in a camera with a lower latency, that uses less power, has reduced bandwidth, and higher dynamic range. Such cameras offer many potential advantages for on-line, autonomous, robots; however, the sensor data do not directly integrate with current "image-based" frameworks and software libraries. The iCub robot uses Yet Another Robot Platform (YARP) as middleware to provide modular processing and connectivity to sensors and actuators. This paper introduces a library that incorporates an event-based framework into the YARP architecture, allowing event cameras to be used with the iCub (and other YARP-based) robots. We describe the philosophy and methods for structuring events to facilitate processing, while maintaining low-latency and real-time operation. We also describe several processing modules made available open-source, and three example demonstrations that can be run on the neuromorphic iCub.

show abstract

“…The motion segmentation problem has also been formulated as one of determining salient regions in spatiotemporal data. Rea et al (2013) implemented a selective saliency model on the iCub platform (Metta et al, 2008), using multiple bottom-up feature maps responsible for contrast, orientation and motion. Serrano-Gotarredona et al (2009) illustrated a parallel very large scale integrated (VLSI) system using the address-event representation (AER) framework, called convolution AER vision architecture for real-time systems (CAVIAR), for object recognition and tracking.…”

Section: Introductionmentioning

confidence: 99%

A Saccade Based Framework for Real-Time Motion Segmentation Using Event Based Vision Sensors

et al. 2017

View full text Add to dashboard Cite

Motion segmentation is a critical pre-processing step for autonomous robotic systems to facilitate tracking of moving objects in cluttered environments. Event based sensors are low power analog devices that represent a scene by means of asynchronous information updates of only the dynamic details at high temporal resolution and, hence, require significantly less calculations. However, motion segmentation using spatiotemporal data is a challenging task due to data asynchrony. Prior approaches for object tracking using neuromorphic sensors perform well while the sensor is static or a known model of the object to be followed is available. To address these limitations, in this paper we develop a technique for generalized motion segmentation based on spatial statistics across time frames. First, we create micromotion on the platform to facilitate the separation of static and dynamic elements of a scene, inspired by human saccadic eye movements. Second, we introduce the concept of spike-groups as a methodology to partition spatio-temporal event groups, which facilitates computation of scene statistics and characterize objects in it. Experimental results show that our algorithm is able to classify dynamic objects with a moving camera with maximum accuracy of 92%.

show abstract

Event-driven visual attention for the humanoid robot iCub

Cited by 35 publications

References 38 publications

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

A Saccade Based Framework for Real-Time Motion Segmentation Using Event Based Vision Sensors

Contact Info

Product

Resources

About