Editorial: Understanding and Bridging the Gap Between Neuromorphic Computing and Machine Learning

Deng, Lei; Tang, Huajin; Roy, Kaushik

doi:10.3389/fncom.2021.665662

Cited by 14 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The artificial synapses must have nearly zero nonlinearity and asymmetricity in conductance modulation for neuromorphic applications. There are various types of ML models of neural networks, including ANN, deep neural networks, and convolutional neural networks in neuromorphic computing . These models mimic the strengthening and weakening of the connection between pre- and postsynaptic neurons by the sum of synaptic weight.…”

Section: Resultsmentioning

confidence: 99%

Electrochemical Analysis of Ion Effects on Electrolyte-Gated Synaptic Transistor Characteristics

Lee,

Cho,

Jin

et al. 2024

ACS Nano

View full text Add to dashboard Cite

Electrolyte-gated transistors (EGTs) are promising candidates as artificial synapses owing to their precise conductance controllability, quick response times, and especially their low operating voltages resulting from ion-assisted signal transmission. However, it is still vague how ion-related physiochemical elements and working mechanisms impact synaptic performance. Here, to address the unclear correlations, we suggest a methodical approach based on electrochemical analysis using poly(ethylene oxide) EGTs with three alkali ions: Li + , Na + , and K + . Cyclic voltammetry is employed to identify the kind of electrochemical reactions taking place at the channel/ electrolyte interface, which determines the nonvolatile memory functionality of the EGTs. Additionally, using electrochemical impedance spectroscopy and qualitative analysis of electrolytes, we confirm that the intrinsic properties of electrolytes (such as crystallinity, solubility, and ion conductivity) and ion dynamics ultimately define the linearity/symmetricity of conductance modulation. Through simple but systematic electrochemical analysis, these results offer useful insights for the selection of components for high-performing artificial synapses.

show abstract

Section: Resultsmentioning

confidence: 99%

Electrochemical Analysis of Ion Effects on Electrolyte-Gated Synaptic Transistor Characteristics

Lee,

Cho,

Jin

et al. 2024

ACS Nano

View full text Add to dashboard Cite

show abstract

“…Through this direct method, ANN-converted SNNs typically need thousands of time steps to represent the information encoded in the spike train for completing a single inference, in addition to the drop in accuracy caused by imposing constraints on the source ANN (Rathi et al 2020;Lee et al 2020a). This is quite a large breakdown from the other SNNs, leading to significant latency and energy consumption opposite to the original purpose (Singh et al 2020;Deng, Tang, and Roy 2021;Lee et al 2020b;Zhang et al 2020).…”

Section: Background and Related Workmentioning

confidence: 99%

SpikeConverter: An Efficient Conversion Framework Zipping the Gap between Artificial Neural Networks and Spiking Neural Networks

Liu

Zhao

Chen

et al. 2022

AAAI

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) have recently attracted enormous research interest since their event-driven and brain-inspired structure enables low-power computation. In image recognition tasks, the best results are achieved by SNN so far utilizing ANN-SNN conversion methods that replace activation functions in artificial neural networks~(ANNs) with integrate-and-fire neurons. Compared to source ANNs, converted SNNs usually suffer from accuracy loss and require a considerable number of time steps to achieve competitive accuracy. We find that the performance degradation of converted SNN stems from the fact that the information capacity of spike trains in transferred networks is smaller than that of activation values in source ANN, resulting in less information being passed during SNN inference. To better correlate ANN and SNN for better performance, we propose a conversion framework to mitigate the gap between the activation value of source ANN and the generated spike train of target SNN. The conversion framework originates from exploring an identical relation in the conversion and exploits temporal separation scheme and novel neuron model for the relation to hold. We demonstrate almost lossless ANN-SNN conversion using SpikeConverter for VGG-16, ResNet-20/34, and MobileNet-v2 SNNs on challenging datasets including CIFAR-10, CIFAR-100, and ImageNet. Our results also show that SpikeConverter achieves the abovementioned accuracy across different network architectures and datasets using 32X - 512X fewer inference time-steps than state-of-the-art ANN-SNN conversion methods.

show abstract

“…Deep neural networks have made tremendous progress and become a prevalent tool for performing various cognitive tasks such as object recognition (Simonyan and Zisserman, 2015;Sandler et al, 2018), natural language processing (Devlin et al, 2018;Radford et al, 2019), and self-driving (Nedevschi et al, 2012;Liu et al, 2017), etc. To leverage the capability of deep neural networks in ubiquitous environments requires deployment not only on large-scale computers but also on portable edge devices Deng et al, 2021). However, the increasing complexity of deep neural networks, coupled with data flooding with distributed sensors continuously generates real-time content and places tremendous energy demands on current computing platforms.…”

Section: Introductionmentioning

confidence: 99%

“…To date, shallow SNN structures (i.e., two fully connected layers) have been widely used for classification. However, training highperformance SNNs with competitive classification accuracy and less latency is a nontrivial problem, limiting their scalability in complex applications (Benjamin et al, 2014;Roy et al, 2019;Sengupta et al, 2019;Comsa et al, 2020;Deng et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SSTDP: Supervised Spike Timing Dependent Plasticity for Efficient Spiking Neural Network Training

et al. 2021

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) are a pathway that could potentially empower low-power event-driven neuromorphic hardware due to their spatio-temporal information processing capability and high biological plausibility. Although SNNs are currently more efficient than artificial neural networks (ANNs), they are not as accurate as ANNs. Error backpropagation is the most common method for directly training neural networks, promoting the prosperity of ANNs in various deep learning fields. However, since the signals transmitted in the SNN are non-differentiable discrete binary spike events, the activation function in the form of spikes presents difficulties for the gradient-based optimization algorithms to be directly applied in SNNs, leading to a performance gap (i.e., accuracy and latency) between SNNs and ANNs. This paper introduces a new learning algorithm, called SSTDP, which bridges the gap between backpropagation (BP)-based learning and spike-time-dependent plasticity (STDP)-based learning to train SNNs efficiently. The scheme incorporates the global optimization process from BP and the efficient weight update derived from STDP. It not only avoids the non-differentiable derivation in the BP process but also utilizes the local feature extraction property of STDP. Consequently, our method can lower the possibility of vanishing spikes in BP training and reduce the number of time steps to reduce network latency. In SSTDP, we employ temporal-based coding and use Integrate-and-Fire (IF) neuron as the neuron model to provide considerable computational benefits. Our experiments show the effectiveness of the proposed SSTDP learning algorithm on the SNN by achieving the best classification accuracy 99.3% on the Caltech 101 dataset, 98.1% on the MNIST dataset, and 91.3% on the CIFAR-10 dataset compared to other SNNs trained with other learning methods. It also surpasses the best inference accuracy of the directly trained SNN with 25~32× less inference latency. Moreover, we analyze event-based computations to demonstrate the efficacy of the SNN for inference operation in the spiking domain, and SSTDP methods can achieve 1.3~37.7× fewer addition operations per inference. The code is available at: https://github.com/MXHX7199/SNN-SSTDP.

show abstract

Editorial: Understanding and Bridging the Gap Between Neuromorphic Computing and Machine Learning

Cited by 14 publications

References 22 publications

Electrochemical Analysis of Ion Effects on Electrolyte-Gated Synaptic Transistor Characteristics

Electrochemical Analysis of Ion Effects on Electrolyte-Gated Synaptic Transistor Characteristics

SpikeConverter: An Efficient Conversion Framework Zipping the Gap between Artificial Neural Networks and Spiking Neural Networks

SSTDP: Supervised Spike Timing Dependent Plasticity for Efficient Spiking Neural Network Training

Contact Info

Product

Resources

About