Fully Integrated Spiking Neural Network with Analog Neurons and RRAM Synapses

Valentian, Alexandre; Rummens, François; Vianello, E.; Mesquida, Thomas; Boissac, Capucine Lecat-Mathieu de; Bichler, Olivier; Reita, C.

doi:10.1109/iedm19573.2019.8993431

Cited by 63 publications

(44 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The maximum reached classification accuracy settles close to 85%, which is lower than the best results on theoretical SNNs (Diehl and Cook, 2015 ). However, as stated above, the aim of the present work is to test SNNs constituents and architectures that can be possibly realized in hybrid CMOS/memristor technology (Valentian et al, 2019 ; Regev et al, 2020 ). As discussed in the Methods section, the inclusion of a homeostatic rule, which is of difficult hardware implementation, would recover a classification accuracy close to the best state of the art results, as demonstrated by Querlioz et al ( 2015 ) with the same network as the one implemented in this work.…”

Section: Resultsmentioning

confidence: 99%

“…In this paper, we aim at moving the first steps toward the optimization of the training of a SNN through system-level simulations as a function of various experimentally-inspired memristive dynamics. Neuron model, training protocol, and architecture are also compatible with a hardware implementation in CMOS technology, as in the silicon chip described in Valentian et al ( 2019 ) and Regev et al ( 2020 ). The investigated memristive dynamics include linear and non-linear evolution bounded within extreme maximum and minimum values, as well as a non-linear evolution asymptotically approaching the boundary values (details are reported below).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

Brivio

Vianello

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are a computational tool in which the information is coded into spikes, as in some parts of the brain, differently from conventional neural networks (NNs) that compute over real-numbers. Therefore, SNNs can implement intelligent information extraction in real-time at the edge of data acquisition and correspond to a complementary solution to conventional NNs working for cloud-computing. Both NN classes face hardware constraints due to limited computing parallelism and separation of logic and memory. Emerging memory devices, like resistive switching memories, phase change memories, or memristive devices in general are strong candidates to remove these hurdles for NN applications. The well-established training procedures of conventional NNs helped in defining the desiderata for memristive device dynamics implementing synaptic units. The generally agreed requirements are a linear evolution of memristive conductance upon stimulation with train of identical pulses and a symmetric conductance change for conductance increase and decrease. Conversely, little work has been done to understand the main properties of memristive devices supporting efficient SNN operation. The reason lies in the lack of a background theory for their training. As a consequence, requirements for NNs have been taken as a reference to develop memristive devices for SNNs. In the present work, we show that, for efficient CMOS/memristive SNNs, the requirements for synaptic memristive dynamics are very different from the needs of a conventional NN. System-level simulations of a SNN trained to classify hand-written digit images through a spike timing dependent plasticity protocol are performed considering various linear and non-linear plausible synaptic memristive dynamics. We consider memristive dynamics bounded by artificial hard conductance values and limited by the natural dynamics evolution toward asymptotic values (soft-boundaries). We quantitatively analyze the impact of resolution and non-linearity properties of the synapses on the network training and classification performance. Finally, we demonstrate that the non-linear synapses with hard boundary values enable higher classification performance and realize the best trade-off between classification accuracy and required training time. With reference to the obtained results, we discuss how memristive devices with non-linear dynamics constitute a technologically convenient solution for the development of on-line SNN training.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

Brivio

Vianello

et al. 2021

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…At present, Backprop has already exploited for offline training of moderate size memristive neural networks (Valentian et al, 2019 ). Backpropagation based on online training schemes has also been implemented in several memristive deep learning accelerators (Li et al, 2018a ; Wang et al, 2019d ; Yao et al, 2020 ), showing great success of memristive array on accelerating the deep learning training and adaptive to some device non-ideal characteristics.…”

Section: Memristive Devices and Computingmentioning

confidence: 99%

“…The first one is the co-integration of non-volatile memristive devices with some peripheral circuits (Hirtzlin et al, 2020 ) and to implement some logic and multiply-and-accumulate (MAC) operations (Chen et al, 2019 ), which reaches the maturity with the demonstration of a fully co-integrated SNN with analog neurons and memristive synapses (Valentian et al, 2019 ). The second phase is the co-integration of different technologies.…”

Section: Memristive Devices and Computingmentioning

confidence: 99%

Adaptive Extreme Edge Computing for Wearable Devices

et al. 2021

View full text Add to dashboard Cite

Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy. Due to the widespread of sensors in pervasive and distributed networks, power consumption, processing speed, and system adaptation are vital in future smart wearable devices. The visioning and forecasting of how to bring computation to the edge in smart sensors have already begun, with an aspiration to provide adaptive extreme edge computing. Here, we provide a holistic view of hardware and theoretical solutions toward smart wearable devices that can provide guidance to research in this pervasive computing era. We propose various solutions for biologically plausible models for continual learning in neuromorphic computing technologies for wearable sensors. To envision this concept, we provide a systematic outline in which prospective low power and low latency scenarios of wearable sensors in neuromorphic platforms are expected. We successively describe vital potential landscapes of neuromorphic processors exploiting complementary metal-oxide semiconductors (CMOS) and emerging memory technologies (e.g., memristive devices). Furthermore, we evaluate the requirements for edge computing within wearable devices in terms of footprint, power consumption, latency, and data size. We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices.

show abstract

“…The integration of CMOS technology with that of the emerging devices has been demonstrated for non-volatile filamentary switches [147] already at a commercial level [148]. There have also been some efforts in combining CMOS and memristor technologies to design supervised local error-based learning circuits using only one network layer by exploiting the properties of memristive devices [143], [149], [150].…”

Section: B Towards Edge Processing For Biomedical Applications With Neuromorphic Processorsmentioning

confidence: 99%

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

Azghadi

Lammie

Eshraghian

et al. 2020

IEEE Trans. Biomed. Circuits Syst.

152

View full text Add to dashboard Cite

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.

show abstract

Fully Integrated Spiking Neural Network with Analog Neurons and RRAM Synapses

Cited by 63 publications

References 6 publications

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

Adaptive Extreme Edge Computing for Wearable Devices

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

Contact Info

Product

Resources

About