Neuromorphic Event-Based Spatio-temporal Attention using Adaptive Mechanisms

Abstract

Event cameras (or silicon retinas) represent a new kind of sensor that measure pixel-wise changes in brightness and output asynchronous events accordingly. This novel technology allows for a sparse and energy-efficient recording and storage of visual information. While this type of data is sparse by definition, the event flow can be very high, up to 25M events per second, which requires significant processing resources to handle and therefore impedes embedded applications. Neuromorphic computer vision and event sensor based applications are receiving an increasing interest from the computer vision community (classification, detection, tracking, segmentation, etc.), especially for robotics or autonomous driving scenarios. Downscaling event data is an important feature in a system, especially if embedded, so as to be able to adjust the complexity of data to the available resources such as processing capability and power consumption. To the best of our knowledge, this works is the first attempt to formalize event data downscaling. In order to study the impact of spatial resolution downscaling, we compare several features of the resulting data, such as the total number of events, event density, information entropy, computation time and optical consistency as assessment criteria. Our code is available online at https://github.com/amygruel/EvVisu.

Advancements in Artificial Intelligence Circuits and Systems (AICAS)

“…This has significantly accelerated progress in diverse fields like natural language processing, where the subtleties of human language are decoded and utilized; image recognition, which now goes beyond mere patterns to interpret context and meaning; and semantic computing, where the interpretation of data becomes as important as the data themselves. Through these monumental advancements, AICAS is not only enhancing current computational methodologies but also paving the way for a future where the boundary between human cognition and machine intelligence becomes increasingly seamless [124][125][126].…”

Object Detection for Embedded Systems Using Tiny Spiking Neural Networks: Filtering Noise Through Visual Attention

“…In our experiments, we chose S = 5, resulting in 16 × 12 individual patches. These patches of input spikes are sent to the next layer, with each patch connected to one neuron via excitatory synapses, in a manner similar to the ROI layer described by Gruel et al [9]. It should be mentioned that no Leaky-Integrate-and-Fire (LIF) neurons have to be simulated on SpiNNaker here: we are merely making a convolution over the input window which is simply a spike source without membrane dynamics.…”

“…Both lateral excitation and inhibition mechanisms are used respectively in order to create smoother areas of activation and to force the layer activity on specific parts of the input. Each neuron is connected to its neighbourhood via excitatory synapses as described by Acharya et al [1], in addition to being connected to all other neurons via exponential inhibitory connections as described by Gruel et al [9] and by Eq 2:…”

“…Unlike traditional approaches that require pretraining the neural network, the methodology presented in this paper does not involve a separate training phase. Instead, synaptic weights are dynamically adjusted during inference, in a manner similar to what is described by Gruel et al [9]. These authors introduced various attentional mechanisms, enabling their network to focus solely on specific areas of the input.…”

Embedded neuromorphic attention model leveraging a novel low-power heterogeneous platform

“…Only a few models genuinely take advantage of the intrinsic dynamics of SNNs and the uniqueness of event data. In particular, Renner et al [4] make use of the mathematical model of Dynamic Neural Field [5] as a soft Winner-Takes-All (WTA) to implement saliency tracking of pre-activated objects; Gruel et al [6] implement a similar mechanism with a significantly low number of neurons and connections.…”