Brain-inspired methods for achieving robust computation in heterogeneous mixed-signal neuromorphic processing systems

Abstract

Neuromorphic processing systems implementing spiking neural networks with mixed signal analog/digital electronic circuits and/or memristive devices represent a promising technology for edge computing applications that require low power, low latency, and that cannot connect to the cloud for off-line processing, either due to lack of connectivity or for privacy concerns. However, these circuits are typically noisy and imprecise, because they are affected by device-to-device variability, and operate with extremely small currents. So achieving reliable computation and high accuracy following this approach is still an open challenge that has hampered progress on the one hand and limited widespread adoption of this technology on the other. By construction, these hardware processing systems have many constraints that are biologically plausible, such as heterogeneity and non-negativity of parameters. More and more evidence is showing that applying such constraints to artificial neural networks, including those used in artificial intelligence, promotes robustness in learning and improves their reliability. Here we delve even more in neuroscience and present network-level brain-inspired strategies that further improve reliability and robustness in these neuromorphic systems: we quantify, with chip measurements, to what extent population averaging is effective in reducing variability in neural responses, we demonstrate experimentally how the neural coding strategies of cortical models allow silicon neurons to produce reliable signal representations, and show how to robustly implement essential computational primitives, such as selective amplification, signal restoration, working memory, and relational networks, exploiting such strategies. We argue that these strategies can be instrumental for guiding the design of robust and reliable ultra-low power electronic neural processing systems implemented using noisy and imprecise computing substrates such as subthreshold neuromorphic circuits and emerging memory technologies.

DenRAM: neuromorphic dendritic architecture with RRAM for efficient temporal processing with delays

“…2 f) 55 . In-line with approaches that propose to exploit variability and heterogeneity in neuromorphic circuits for achieving robust computation 56 , DenRAM takes advantage of such heterogeneity, by providing a population of analog delay elements per input channel, thus enriching the circuit with a delay spectrum that can be tuned by the weight values.…”

Robust compression and detection of epileptiform patterns in ECoG using a real-time spiking neural network hardware framework

“…Due to their analog nature, the neuron circuits exhibit a variability induced by circuit device mismatch factors that arise during circuit fabrication 24 . Although circuit parameters are shared between all neurons in the same DYNAP-SE core, the device mismatch induced variability produces a distribution of parameters, with shared mean values, but with a coefficient of variation that can be as large as 20% 25 . Although device mismatch is typically perceived as a limitation in analog computation, in our analysis the inherent neural variability is beneficial since it allows processing the incoming ADM pulses with an ensemble of heterogeneous neurons, which has been shown to improve the information encoding and classification accuracy 26 , 27 .…”

Spike-based local synaptic plasticity: a survey of computational models and neuromorphic circuits

“…Many of the models considered rely on exponentially decaying traces. By operating the CMOS circuits in the sub-threshold regime, this exponential dependency is given by the physical substrate of transistors showing an exponential relationship between current and voltage [10,147].…”