Learning to Reproduce Visually Similar Movements by Minimizing Event-Based Prediction Error

Kaiser, Jacques; Melbaum, Svenja; Tieck, J. Camilo Vasquez; Roennau, Arne; Butz, Martin V.; Dillmann, Rüdiger

doi:10.1109/biorob.2018.8487959

Cited by 15 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the recent advances in backpropagation-like learning rules for SNN as in Kaiser et al (2019), we can learn different motion types for different tasks in same network, and start them with different go-cues. We also want to integrate event-based vision to this system to get the target and drive the adaptation as in Kaiser et al (2016), and to explore learning by demonstration as in Kaiser et al (2018). We work on extending this work form pointing to a given target to perform there a grasping or tool manipulation task.…”

Section: Discussionmentioning

confidence: 99%

Generating Pointing Motions for a Humanoid Robot by Combining Motor Primitives

et al. 2019

Self Cite

View full text Add to dashboard Cite

The human motor system is robust, adaptive and very flexible. The underlying principles of human motion provide inspiration for robotics. Pointing at different targets is a common robotics task, where insights about human motion can be applied. Traditionally in robotics, when a motion is generated it has to be validated so that the robot configurations involved are appropriate. The human brain, in contrast, uses the motor cortex to generate new motions reusing and combining existing knowledge before executing the motion. We propose a method to generate and control pointing motions for a robot using a biological inspired architecture implemented with spiking neural networks. We outline a simplified model of the human motor cortex that generates motions using motor primitives. The network learns a base motor primitive for pointing at a target in the center, and four correction primitives to point at targets up, down, left and right from the base primitive, respectively. The primitives are combined to reach different targets. We evaluate the performance of the network with a humanoid robot pointing at different targets marked on a plane. The network was able to combine one, two or three motor primitives at the same time to control the robot in real-time to reach a specific target. We work on extending this work from pointing to a given target to performing a grasping or tool manipulation task. This has many applications for engineering and industry involving real robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Generating Pointing Motions for a Humanoid Robot by Combining Motor Primitives

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…The system is capable of learning the temporal structure of space-time events, adapt to multiple scales and is able to predict future events in a video sequence. Using a DVS camera, a method is presented in Kaiser et al (2018) to learn movements from visual predictions. The proposed method consists of two phases.…”

Section: Predictionmentioning

confidence: 99%

“…Vision (Klein et al, 2015), predator robot (Moeys et al, 2016a), robot goalies (Becanovic et al, 2002Delbruck and Lichtsteiner, 2007;Delbruck and Lang, 2013), humanoid robot (Rea et al, 2013) Algorithms Algorithms Mapping (Pérez-Carrasco et al, 2013), filtering (Ieng et al, 2014;Bidegaray-Fesquet, 2015), lifetime estimation (Mueggler et al, 2015b), classification (Li et al, 2018), compression (Brandli et al, 2014;Doutsi et al, 2015;Bi et al, 2018), prediction (Gibson et al, 2014b;Kaiser et al, 2018), high-speed frame capturing (Liu et al, 2017b;Pan et al, 2018), spiking neural networks (Dhoble et al, 2012;Stromatias et al, 2017), data transmission (Corradi and Indiveri, 2015), matching (Moser, 2015) hybrid methods Indiveri, 2011a,b, 2012;Weikersdorfer et al, 2014;Leow and Nikolic, 2015), fusion (Akolkar et al, 2015b;Rios-Navarro et al, 2015;Neil and Liu, 2016) Feature extraction Vehicle detection (Bichler et al, 2011(Bichler et al, , 2012, gesture recognition (Ahn, 2012), robot vision (Lagorce et al, 2013), hardware implementation (del Campo et al, 2013;Yousefzadeh et al, 2015;Hoseini and Linares-Barranco, 2018), optical flow (Koeth et al, 2013;Clady et al, 2017;Zhu et al, 2017a), feature extraction algorithms (Lagorce et al, 2015a;…”

Section: Vision and Attentionmentioning

confidence: 99%

Event-Based Sensing and Signal Processing in the Visual, Auditory, and Olfactory Domain: A Review

Tayarani-Najaran

Schmuker

2021

Front. Neural Circuits

View full text Add to dashboard Cite

The nervous systems converts the physical quantities sensed by its primary receptors into trains of events that are then processed in the brain. The unmatched efficiency in information processing has long inspired engineers to seek brain-like approaches to sensing and signal processing. The key principle pursued in neuromorphic sensing is to shed the traditional approach of periodic sampling in favor of an event-driven scheme that mimicks sampling as it occurs in the nervous system, where events are preferably emitted upon the change of the sensed stimulus. In this paper we highlight the advantages and challenges of event-based sensing and signal processing in the visual, auditory and olfactory domains. We also provide a survey of the literature covering neuromorphic sensing and signal processing in all three modalities. Our aim is to facilitate research in event-based sensing and signal processing by providing a comprehensive overview of the research performed previously as well as highlighting conceptual advantages, current progress and future challenges in the field.

show abstract

“…We note that a similar mechanism could be integrated in a robotic head as the one used in this paper to perform actual eye movements (see Figure 2). However, an additional mechanism to discard events resulting of the ego-motion would be required, which could be based on visual prediction [11], [13], [14].…”

Section: Covert Attention Windowmentioning

confidence: 99%

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Kaiser

Friedrich

Tieck

et al. 2020

2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)

Self Cite

View full text Add to dashboard Cite

Spike-based communication between biological neurons is sparse and unreliable. This enables the brain to process visual information from the eyes efficiently. Taking inspiration from biology, artificial spiking neural networks coupled with silicon retinas attempt to model these computations. Recent findings in machine learning allowed the derivation of a family of powerful synaptic plasticity rules approximating backpropagation for spiking networks. Are these rules capable of processing real-world visual sensory data? In this paper, we evaluate the performance of Event-Driven Random Back-Propagation (eRBP) at learning representations from event streams provided by a Dynamic Vision Sensor (DVS). First, we show that eRBP matches state-of-the-art performance on the DvsGesture dataset with the addition of a simple covert attention mechanism. By remapping visual receptive fields relatively to the center of the motion, this attention mechanism provides translation invariance at low computational cost compared to convolutions. Second, we successfully integrate eRBP in a real robotic setup, where a robotic arm grasps objects according to detected visual affordances. In this setup, visual information is actively sensed by a DVS mounted on a robotic head performing microsaccadic eye movements. We show that our method classifies affordances within 100ms after microsaccade onset, which is comparable to human performance reported in behavioral study. Our results suggest that advances in neuromorphic technology and plasticity rules enable the development of autonomous robots operating at high speed and low energy consumption.

show abstract

Learning to Reproduce Visually Similar Movements by Minimizing Event-Based Prediction Error

Cited by 15 publications

References 24 publications

Generating Pointing Motions for a Humanoid Robot by Combining Motor Primitives

Generating Pointing Motions for a Humanoid Robot by Combining Motor Primitives

Event-Based Sensing and Signal Processing in the Visual, Auditory, and Olfactory Domain: A Review

Embodied Neuromorphic Vision with Continuous Random Backpropagation

Contact Info

Product

Resources

About