In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

Shrestha, Amar; Fang, Haowen; Rider, Daniel Patrick; Mei, Zaidao; Qiu, Qinru

doi:10.1109/dac18074.2021.9586323

Cited by 23 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, the performances of the artificial neural networks can be evaluated through their recognition capability for several types of pattern images. [68][69][70] To confirm the capability of the developed PVA-based memristor for constructing the complex neural network, we conducted the numerical simulation of SPICE for the handwritten digit pattern recognition based on a data set of the Modified National Institute of Standards and Technology (MNIST), [71,72] as shown in Figure 5e. In the simulation, 60 000 images for learning and 10 000 images for classifying tests were utilized, and the pixel of each image had 256 levels for grayscale.…”

Section: Resultsmentioning

confidence: 99%

“…In addition, when the current‐sensing resistor of 1 mΩ was utilized, the developed hardware neural networks classified digit images by consuming about 255 pJ, which is greatly more efficient than that of the von Neumann counterparts. [ 70 ] This implies that the developed PVA‐based memristor with biodegradability and mechanical flexibility can be used as a synaptic device in the energy efficient hardware neural networks with high integration density.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Biodegradable and Flexible Polymer‐Based Memristor Possessing Optimized Synaptic Plasticity for Eco‐Friendly Wearable Neural Networks with High Energy Efficiency

Kim

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Biodegradable and Flexible Polymer‐Based Memristor Possessing Optimized Synaptic Plasticity for Eco‐Friendly Wearable Neural Networks with High Energy Efficiency

Kim

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…In general, the performances of the artificial neural networks can be evaluated through their recognition capability for several types of pattern images (Feng et al, 2021;Kim et al, 2021b;Shrestha et al, 2021). To confirm the capability of the developed PVA-based memristor for constructing the complex neural network, we conducted the numerical simulation of SPICE for the handwritten digit pattern recognition based on a dataset of the Modified National Institute of Standards and Technology (MNIST) Wang et al, 2020), as shown in Figure 5e.…”

Section: Resultsmentioning

confidence: 99%

“…In the neural network based on the developed memristor, the pattern recognition accuracy was about 92 % after training 50 epochs, which is highly close to that of the ideal software system (see Figure 5i). In addition, when the current-sensing resistor of 1 mΩ was utilized, the developed hardware neural networks classified digit images by consuming about 255 pJ, which is greatly more efficient than that of the von Neumann counterparts (Shrestha et al, 2021). This implies that the developed PVA-based memristor with biodegradability and mechanical flexibility can be used as a synaptic device in the energy efficient hardware neural networks with high integration density.…”

Section: Resultsmentioning

confidence: 99%

Biodegradable and Flexible Polymer Based Memristor Possessing Optimized Synaptic Plasticity for Eco-Friendly Wearable Neural Networks with High Energy Efficiency

Oh¹,

Kim²,

Kim³

et al. 2022

Preprint

View full text Add to dashboard Cite

Organic memristors are promising candidates for the flexible synaptic components of wearable intelligent systems. With heightened concerns for the environment, considerable effort has been made to develop organic transient memristors to realize eco-friendly flexible neural networks. However, in the transient neural networks, achieving flexible memristors with bio-realistic synaptic plasticity for energy efficient learning processes is still challenging. Here, we demonstrate a biodegradable and flexible polymer based memristor, suitable for the spike-dependent learning process. An electrochemical metallization phenomenon for the conductive nanofilament growth in a polymer medium of poly (vinyl alcohol) (PVA) is analyzed and a PVA based transient and flexible artificial synapse is developed. The developed device exhibits superior biodegradability and stable mechanical flexibility due to the high water solubility and excellent tensile strength of the PVA film, respectively. In addition, the developed flexible memristor is operated as a reliable synaptic device with optimized synaptic plasticity, which is ideal for artificial neural networks with the spike-dependent operations. The developed device is found to be effectively served as a reliable synaptic component with high energy efficiency in practical neural networks. This novel strategy for developing transient and flexible artificial synapses can be a fundamental platform for realizing eco-friendly wearable intelligent systems. Corresponding author(s) Email: sinhlee@knu.ac.kr

show abstract

“…To our knowledge, this work is the first to show an SNN implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. Other on-chip learning approaches so far either use feedback alignment [67], forward propagation of errors [32] or single layer training [25,27,29,68,69]. Compared to an equivalent implementation on a GPU, there is no loss in accuracy, but there are about two orders of magnitude power savings in the case of small batch sizes which are more realistic for edge computing settings.…”

Section: Significancementioning

confidence: 99%

The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

Renner¹,

Sheldon²,

Zlotnik³

et al. 2021

Preprint

View full text Add to dashboard Cite

The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, most modern machine learning algorithms are not neurophysiologically plausible and thus are not directly implementable in neuromorphic hardware. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on pulse-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. This implementation shows a path for using massively parallel, low-power, low-latency neuromorphic processors in modern deep learning applications.

show abstract

In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

Cited by 23 publications

References 11 publications

Biodegradable and Flexible Polymer‐Based Memristor Possessing Optimized Synaptic Plasticity for Eco‐Friendly Wearable Neural Networks with High Energy Efficiency

Biodegradable and Flexible Polymer‐Based Memristor Possessing Optimized Synaptic Plasticity for Eco‐Friendly Wearable Neural Networks with High Energy Efficiency

Biodegradable and Flexible Polymer Based Memristor Possessing Optimized Synaptic Plasticity for Eco-Friendly Wearable Neural Networks with High Energy Efficiency

The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

Contact Info

Product

Resources

About