On Practical Issues for Stochastic STDP Hardware With 1-bit Synaptic Weights

Yousefzadeh, Amirreza; Stromatias, Evangelos; Soto, Miguel; Serrano-Gotarredona, Teresa; Linares-Barranco, B.

doi:10.3389/fnins.2018.00665

Cited by 60 publications

(60 citation statements)

References 80 publications

(156 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This intrinsic probabilistic property of memristive devices can be exploited for implementing stochastic learning in neuromorphic architectures 43,44,48,[64][65][66] , which in turn can be used to implement faithful models of biological cortical microcircuits 67,68 , solve memory capacity and classification problems in artificial neural network applications 69,70 , and reduce the network sensitivity to their variability 43 . Recent results on stochastic learning modulated by regularization mechanisms, such as homeostasis or intrinsic plasticity 44,[71][72][73] , present an excellent potential for exploiting the features memristive devices, even when restricted to binary values. e. Don't (hard) limit your devices.…”

mentioning

confidence: 99%

A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

Chicca

Indiveri

2020

Applied Physics Letters

View full text Add to dashboard Cite

The development of memristive device technologies has reached a level of maturity to enable the design of complex and large-scale hybrid memristive-CMOS neural processing systems. These systems offer promising solutions for implementing novel in-memory computing architectures for machine learning and data analysis problems. We argue that they are also ideal building blocks for the integration in neuromorphic electronic circuits suitable for ultra-low power brain-inspired sensory processing systems, therefore leading to the innovative solutions for always-on edge-computing and Internet-of-Things (IoT) applications. Here we present a recipe for creating such systems based on design strategies and computing principles inspired by those used in mammalian brains. We enumerate the specifications and properties of memristive devices required to support always-on learning in neuromorphic computing system and to minimize their power consumption. Finally, we discuss in what cases such neuromorphic systems can complement conventional processing ones and highlight the importance of exploiting the physics of both the memristive devices and of the CMOS circuits interfaced to them.Neuromorphic computing has recently received considerable attention as a discipline that can offer promising technological solutions for implementing power-and size-efficient sensory-processing, learning, and Artificial Intelligence (AI) applications 1-5 , especially in cases in which the computing system has to operate autonomously "at the edge", i.e., without having to connect to powerful (but power hungry) server farms in the "cloud". The term "neuromorphic" was originally coined in the early 90's by Carver Mead to refer to mixed signal analog/digital Very Large Scale Integration (VLSI) computing systems based on the organizing principles used by the biological nervous systems 6 . In that context, "neuromorphic engineering" emerged as an interdisciplinary research field deeply rooted in biology that focused on building electronic neural processing systems by exploiting the physics of silicon to directly "emulate" the bio-physics of real neurons and synapses. More recently the definition of the term "neuromorphic" has been extended in two additional directions: on one hand to describe more generic spike-based processing systems engineered to "simulate" spiking neural networks for the exploration of large-scale computational neuroscience models 7-9 ; and on the other hand to describe dedicated electronic neural architectures that make use of both electronic Complementary Metal-Oxide Semiconductor (CMOS) circuits and memristive devices to implement neuron and synapse circuits 10,11 .Another recent and very promising trend in developing dedicated hardware architectures for building accelerated simulators of artificial neural networks is related to the field of machine learning and AI 12,13 . The types of neural networks being proposed within this context are only loosely inspired by biology, are aimed at high accuracy pattern recognition based on large...

show abstract

mentioning

confidence: 99%

A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

Chicca

Indiveri

2020

Applied Physics Letters

View full text Add to dashboard Cite

show abstract

“…However, as it also follows from a bottom-up design approach, scaling S-SDSP to more complex tasks is not straightforward as it would require going beyond single-layer training. Further research is required to leverage brain-inspired local plasticity primitives with multi-layer networks for online learning on real-world tasks, as highlighted by the recent S-STDP study by Yousefzadeh et al [37].…”

Section: B S-sdsp Online Learningmentioning

confidence: 99%

MorphIC: A 65-nm 738k-Synapse/mm$^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning

Frenkel

Legat

Bol

2019

IEEE Trans. Biomed. Circuits Syst.

142

View full text Add to dashboard Cite

Recent trends in the field of neural network accelerators investigate weight quantization as a means to increase the resource-and power-efficiency of hardware devices. As full on-chip weight storage is necessary to avoid the high energy cost of off-chip memory accesses, memory reduction requirements for weight storage pushed toward the use of binary weights, which were demonstrated to have a limited accuracy reduction on many applications when quantization-aware training techniques are used. In parallel, spiking neural network (SNN) architectures are explored to further reduce power when processing sparse eventbased data streams, while on-chip spike-based online learning appears as a key feature for applications constrained in power and resources during the training phase. However, designing power-and area-efficient spiking neural networks still requires the development of specific techniques in order to leverage onchip online learning on binary weights without compromising the synapse density. In this work, we demonstrate MorphIC, a quadcore binary-weight digital neuromorphic processor embedding a stochastic version of the spike-driven synaptic plasticity (S-SDSP) learning rule and a hierarchical routing fabric for large-scale chip interconnection. The MorphIC SNN processor embeds a total of 2k leaky integrate-and-fire (LIF) neurons and more than two million plastic synapses for an active silicon area of 2.86mm 2 in 65nm CMOS, achieving a high density of 738k synapses/mm 2 . MorphIC demonstrates an order-of-magnitude improvement in the area-accuracy tradeoff on the MNIST classification task compared to previously-proposed SNNs, while having no penalty in the energy-accuracy tradeoff.

show abstract

“…Many researchers aim to train SNNs by using bio-inspired learning rules [18] [19] [20] [21] and try to model the current understanding of the brain. Other researchers exploit the algorithmic advances in deep learning to train an ANN and study methods to convert a pre-trained ANN to an SNN architecture that is more suited for mapping to asynchronous The green neurons are the ones that received a spike and should be updated and the red neurons are the ones that fired after receiving a spike.…”

Section: * Amirreza Yousefzadeh and Mina A Khoei Contributed Equally mentioning

confidence: 99%

Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity

Yousefzadeh

Serrano-Gotarredona

Linares-Barranco

et al. 2019

IEEE J. Emerg. Sel. Topics Circuits Syst.

Self Cite

View full text Add to dashboard Cite

Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.

show abstract

On Practical Issues for Stochastic STDP Hardware With 1-bit Synaptic Weights

Cited by 60 publications

References 80 publications

A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

MorphIC: A 65-nm 738k-Synapse/mm$^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning

Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity

Contact Info

Product

Resources

About