Embedded neuromorphic vision for humanoid robots

Bartolozzi, Chiara; Clercq, Charles De; Fasnacht, Daniel; Indiveri, Giacomo; Hofstätter, Michael; Metta, Giorgio

doi:10.1109/cvprw.2011.5981834

Cited by 43 publications

(30 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The neuromorphic iCub (Bartolozzi et al, 2011) is a humanoid robot that has a vision system comprised of two event cameras. The iCub robot is supported, in software, by the Yet Another Robot Platform (YARP) middleware (Metta et al, 2006), upon which the iCub low-level and application-level modules have matured using standard cameras, and also utilized other freely available algorithms (e.g., using OpenCV).…”

Section: Introductionmentioning

confidence: 99%

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

et al. 2018

Self Cite

View full text Add to dashboard Cite

Citation:Glover A, Vasco V, Iacono M and Bartolozzi C (2018) Event-driven (ED) cameras are an emerging technology that sample the visual signal based on changes in the signal magnitude, rather than at a fixed-rate over time. The change in paradigm results in a camera with a lower latency, that uses less power, has reduced bandwidth, and higher dynamic range. Such cameras offer many potential advantages for on-line, autonomous, robots; however, the sensor data do not directly integrate with current "image-based" frameworks and software libraries. The iCub robot uses Yet Another Robot Platform (YARP) as middleware to provide modular processing and connectivity to sensors and actuators. This paper introduces a library that incorporates an event-based framework into the YARP architecture, allowing event cameras to be used with the iCub (and other YARP-based) robots. We describe the philosophy and methods for structuring events to facilitate processing, while maintaining low-latency and real-time operation. We also describe several processing modules made available open-source, and three example demonstrations that can be run on the neuromorphic iCub.

show abstract

Section: Introductionmentioning

confidence: 99%

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Vision algorithms have implemented in neural architectures or embedded systems, such as [13], [2], [11]. Other works [23], [10] used spiking neurons for basic obstacle detection and navigation.…”

Section: Introductionmentioning

confidence: 99%

Low-power parallel algorithms for single image based obstacle avoidance in aerial robots

Lenz

Gemici

Saxena

2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-For an aerial robot, perceiving and avoiding obstacles are necessary skills to function autonomously in a cluttered unknown environment. In this work, we use a single image captured from the onboard camera as input, produce obstacle classifications, and use them to select an evasive maneuver. We present a Markov Random Field based approach that models the obstacles as a function of visual features and non-local dependencies in neighboring regions of the image. We perform efficient inference using new low-power parallel neuromorphic hardware, where belief propagation updates are done using leaky integrate and fire neurons in parallel, while consuming less than 1 W of power. In outdoor robotic experiments, our algorithm was able to consistently produce clean, accurate obstacle maps which allowed our robot to avoid a wide variety of obstacles, including trees, poles and fences.

show abstract

“…However most of these applications are demonstrated with stationary sensors. The use of the DVS retinas on a moving robotic head with the degrees of freedom to approximate human-like head and eye movements is rare [4], [5]. The inclusion of event-based silicon cochleas with a silicon retina on a robotic platform is also rare [6].…”

Section: Introductionmentioning

confidence: 99%

Scene stitching with event-driven sensors on a robot head platform

Klein

Conradt

Liu

2015

2015 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

This paper describes a robot head platform which holds a pair of event-based Dynamic Vision Sensor (DVS) retinas and microphones connected to an event-based binaural AEREAR2 VLSI cochlea system. The platform has 6 degrees of freedom (DOF): 2 for the neck, and 2 for each of the DVS retinas. Two applications using this platform are described: the first is image stitching of a scene larger than the field of view of the individual retinas as the head pans and tilts and the second is selective image painting of the local visual scene around spatially displaced sound sources. This platform allows for the investigation of event-driven sensory-action models that use the information from multiple event-based sensor modalities in real-time scenarios. Abstract-This paper describes a robot head platform which holds a pair of event-based Dynamic Vision Sensor (DVS) retinas and microphones connected to an event-based binaural AEREAR2 VLSI cochlea system. The platform has 6 degrees of freedom (DOF): 2 for the neck, and 2 for each of the DVS retinas. Two applications using this platform are described: the first is image stitching of a scene larger than the field of view of the individual retinas as the head pans and tilts and the second is selective image painting of the local visual scene around spatially displaced sound sources. This platform allows for the investigation of event-driven sensory-action models that use the information from multiple event-based sensor modalities in real-time scenarios.

show abstract

Embedded neuromorphic vision for humanoid robots

Cited by 43 publications

References 31 publications

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

The Event-Driven Software Library for YARP—With Algorithms and iCub Applications

Low-power parallel algorithms for single image based obstacle avoidance in aerial robots

Scene stitching with event-driven sensors on a robot head platform

Contact Info

Product

Resources

About