How Late is too Late? A Preliminary Event-based Latency Evaluation

Gava, Luna; Monforte, Marco; Bartolozzi, Chiara; Glover, Arren

doi:10.1109/ebccsp56922.2022.9845622

Cited by 2 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the previous strokes) and only returns visual information about the most recent brushstroke being performed, overcoming the limitation of traditional computer vision approaches to extract individual brushstrokes from complex scenes. Due to their sparse and asynchronous encoding with a stream of events, event-based cameras achieve temporal resolutions of microseconds and ultra-low power consumption [10], making them ideal for robotic applications.…”

Section: Introductionmentioning

confidence: 99%

A neuromorphic electronic artist for robotic painting

Schürmann,

D'Angelo,

Indiveri

et al. 2024

Preprint

View full text Add to dashboard Cite

Recent advances in computer vision and deep learning have led to a surge of interest in the field of AI-generated art, including digital image creation and robot-assisted painting. Traditional painting machines rely on static images and offline processing to incorporate visual feedback into their painting process. However, this approach does not consider the dynamic nature of painting and fails to decompose complex overlapping patterns into individual strokes. As an alternative to frame-based RGB cameras, neuromorphic cameras capture changes in light intensity within a scene via asynchronous event streams, promising to overcome some of the inherent limitations of traditional computer vision techniques. In this project, a robotic system for physical painting is presented which utilizes event-based visual input from a Dynamic Vision Sensor (DVS) camera. To take advantage of the camera's ultra-low latency and sparse encoding, the proposed system also employs event-based information processing, implemented with spiking neural networks on the neuromorphic DynapSE-1 processor. The robotic system receives DVS sensory data which represents the trajectory of a brush stroke and computes the required joint velocities to recreate the stroke with a 6-DOF robotic arm in a closed-loop manner. The controller additionally integrates tactile feedback from a force-torque sensor to dynamically adjust the end-effector’s distance towards the canvas depending on the brush’s deformation. Within the scope of the project, it was further demonstrated how speed information about a perceived brush stroke can be extracted from DVS data. The system was tested in a real-world setting and successfully generated a collection of physical brush strokes. The proposed network is a first step towards a fully spiking robotic controller with the ability to seamlessly incorporate event-based sensory feedback, providing ultra-low latency responsiveness. Beyond its utility in robot-assisted painting, the developed network is applicable to any robotic task requiring real-time adaptive control.

show abstract

Section: Introductionmentioning

confidence: 99%