Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Sengupta, Abhronil; Ye, Yuting; Wang, Robert; Liu, Chiao; Roy, Kaushik

doi:10.3389/fnins.2019.00095

Cited by 875 publications

(724 citation statements)

References 23 publications

Supporting

Mentioning

717

Contrasting

Unclassified

Order By: Relevance

“…From the algorithmic perspective, SNN computing models are a significant shift from the traditional deep non-spiking networks (current de-facto standard) due to the additional time domain encoding of information. Hence, classification accuracies provided by such networks are still limited than their non-spiking counterparts 7 . Further, it is unclear whether emerging neuromorphic devices based on spintronics, resistive memories, phase-change memories would still exhibit multi-level characteristics at aggressively scaled device dimensions (the key characteristic being leveraged in these non-volatile devices).…”

mentioning

confidence: 99%

Stochastic magnetoelectric neuron for temporal information encoding

Yang

Sengupta

2020

Applied Physics Letters

Self Cite

View full text Add to dashboard Cite

Emulating various facets of computing principles of the brain can potentially lead to the development of neurocomputers that are able to exhibit brain-like cognitive capabilities. In this letter, we propose a magnetoelectronic neuron that utilizes noise as a computing resource and is able to encode information over time through the independent control of external voltage signals. We extensively characterize the device operation using simulations and demonstrate its suitability for neuromorphic computing platforms performing temporal information encoding.

show abstract

mentioning

confidence: 99%

Stochastic magnetoelectric neuron for temporal information encoding

Yang

Sengupta

2020

Applied Physics Letters

Self Cite

View full text Add to dashboard Cite

show abstract

“…To compare the energy efficiency of ANN and its equivalent SNN implementation, we follow the convention from NC community and compute the total synaptic operations SynOps that required to perform a certain task (Merolla et al, 2014;Rueckauer et al, 2017;Sengupta et al, 2019). For ANN, the total synaptic operations (Multiply-and-Accumulate (MAC)) per classification is defined as follows…”

Section: Energy Efficiency: Counting Synaptic Operationsmentioning

confidence: 99%

“…Recently, considerable research efforts are devoted to addressing this problem and the resulting learning rules can be broadly categorized into the SNN-to-ANN conversion (Cao et al, 2015;Diehl et al, 2015), back-propagation through time with surrogate gradient (Neftci et al, 2019; and tandem learning . Despite several successful attempts on the large-scale image classification tasks with deep SNNs (Rueckauer et al, 2017;Hu et al, 2018;Sengupta et al, 2019;Wu et al, 2019), their applications to the large-vocabulary continuous ASR (LVCSR) tasks remain unexplored. In this work, we explore an SNN-based acoustic model for LVCSR using a recently proposed tandem learning rule that supports an efficient and rapid inference.…”

Section: Introductionmentioning

confidence: 99%

Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition

Yılmaz

Zhang

et al. 2020

Front. Neurosci.

View full text Add to dashboard Cite

Artificial neural networks (ANN) have become the mainstream acoustic modeling technique for large vocabulary automatic speech recognition (ASR). A conventional ANN features a multi-layer architecture that requires massive amounts of computation. The brain-inspired spiking neural networks (SNN) closely mimic the biological neural networks and can operate on low-power neuromorphic hardware with spike-based computation. Motivated by their unprecedented energyefficiency and rapid information processing capability, we explore the use of SNNs for speech recognition. In this work, we use SNNs for acoustic modeling and evaluate their performance on several large vocabulary recognition scenarios. The experimental results demonstrate competitive ASR accuracies to their ANN counterparts, while require significantly reduced computational cost and inference time. Integrating the algorithmic power of deep SNNs with energy-efficient neuromorphic hardware, therefore, offer an attractive solution for ASR applications running locally on mobile and embedded devices.

show abstract

“…Some recent work on SNN has focussed on converting images from ANN benchmarks to spike trains and then classifying them [16], [17]. While being great pieces of research, we feel that this is fundamentally not a good application for SNN since the original input signal is static and does not change with time.…”

Section: Neuromorphic Benchmarksmentioning

confidence: 99%

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Bose

Acharya

Basu

2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

View full text Add to dashboard Cite

In this paper, we review recent work published over the last 3 years under the umbrella of Neuromorphic engineering to analyze what are the common features among such systems. We see that there is no clear consensus but each system has one or more of the following features:(1) Analog computing (2) Non von-Neumann Architecture and low-precision digital processing (3) Spiking Neural Networks (SNN) with components closely related to biology. We compare recent machine learning accelerator chips to show that indeed analog processing and reduced bit precision architectures have best throughput, energy and area efficiencies. However, pure digital architectures can also achieve quite high efficiencies by just adopting a non von-Neumann architecture. Given the design automation tools for digital hardware design, it raises a question on the likelihood of adoption of analog processing in the near future for industrial designs. Next, we argue about the importance of defining standards and choosing proper benchmarks for the progress of neuromorphic system designs and propose some desired characteristics of such benchmarks. Finally, we show brain-machine interfaces as a potential task that fulfils all the criteria of such benchmarks.

show abstract

Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Cited by 875 publications

References 23 publications

Stochastic magnetoelectric neuron for temporal information encoding

Stochastic magnetoelectric neuron for temporal information encoding

Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition

Is my Neural Network Neuromorphic? Taxonomy, Recent Trends and Future Directions in Neuromorphic Engineering

Contact Info

Product

Resources

About