Early Termination Based Training Acceleration for an Energy-Efficient SNN Processor Design

Choi, Sunghyun; Lew, Dongwoo; Park, Jongsun

doi:10.1109/tbcas.2022.3181808

Cited by 5 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pan et al [42] presented a bit-serial early termination for convolutional neural networks (CNNs). Choi et al [43] presented early termination for spiking neural network (SNN) processors. Although our propositions are in the same spirit, they have been applied in a different context of frequency transforms of neural tensors.…”

Section: Predictive Early Termination By Exploiting Output Sparsitymentioning

confidence: 99%

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

Darabi,

Hashem,

Pan

et al. 2024

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

The edge processing of deep neural networks (DNNs) is becoming increasingly important due to its ability to extract valuable information directly at the data source to minimize latency and energy consumption. Although pruning techniques are commonly used to reduce model size for edge computing, they have certain limitations. Frequency-domain model compression, such as with the Walsh-Hadamard transform (WHT), has been identified as an efficient alternative. However, the benefits of frequency-domain processing are often offset by the increased multiply-accumulate (MAC) operations required. This article proposes a novel approach to an energy-efficient acceleration of frequency-domain neural networks by utilizing analog-domain frequency-based tensor transformations. Our approach offers unique opportunities to enhance computational efficiency, resulting in several high-level advantages, including array microarchitecture with parallelism, analog-to-digital converter (ADC)/digital-to-analog converter (DAC)-free analog computations, and increased output sparsity. Our approach achieves more compact cells by eliminating the need for trainable parameters in the transformation matrix. Moreover, our novel array microarchitecture enables adaptive stitching of cells column-wise and row-wise, thereby facilitating perfect parallelism in computations. Additionally, our scheme enables ADC/DAC-free computations by training against highly quantized matrix-vector products, leveraging the parameter-free nature of matrix multiplications. Another crucial aspect of our design is its ability to handle signed-bit processing for frequencybased transformations. This leads to increased output sparsity and reduced digitization workload. On a 16 × 16 crossbars, for 8-bit input processing, the proposed approach achieves the energy efficiency of 801 tera operations per second per Watt (TOPS/W) without early termination strategy and 2655 TOPS/W with early termination strategy at VDD = 0.85 V for 16-nm predictive technology models (PTM).

show abstract

Section: Predictive Early Termination By Exploiting Output Sparsitymentioning

confidence: 99%

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

Darabi,

Hashem,

Pan

et al. 2024

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

show abstract

“…A conservative estimate of the energy required to train the network is calculated to be 1.6 μJ per training image by assuming the lowest resistance (and thus highest current) for each memristive weight in the network. Although more expensive per training image than some advanced training algorithms that have an energy consumption of 71.3 nJ per image ( Choi et al, 2022 ), the more compact network and quicker learning rate of the dynamical memristive neural network results in an ultralow total training energy of 94 mJ before convergence. Conventional neural networks that use backpropagation require hundreds of epochs to train a network towards convergence, resulting in a total energy consumption of 1.7–6.4 J depending on the emphasis placed on energy efficiency.…”

Section: Stdp-enabled Physical Neural Network With Lifelong Self-lear...mentioning

confidence: 99%

Dynamical memristive neural networks and associative self-learning architectures using biomimetic devices

Zivasatienraj

Doolittle

2023

Front. Neurosci.

View full text Add to dashboard Cite

While there is an abundance of research on neural networks that are “inspired” by the brain, few mimic the critical temporal compute features that allow the brain to efficiently perform complex computations. Even fewer methods emulate the heterogeneity of learning produced by biological neurons. Memory devices, such as memristors, are also investigated for their potential to implement neuronal functions in electronic hardware. However, memristors in computing architectures typically operate as non-volatile memories, either as storage or as the weights in a multiply-and-accumulate function that requires direct access to manipulate memristance via a costly learning algorithm. Hence, the integration of memristors into architectures as time-dependent computational units is studied, starting with the development of a compact and versatile mathematical model that is capable of emulating flux-linkage controlled analog (FLCA) memristors and their unique temporal characteristics. The proposed model, which is validated against experimental FLCA LixNbO2 intercalation devices, is used to create memristive circuits that mimic neuronal behavior such as desensitization, paired-pulse facilitation, and spike-timing-dependent plasticity. The model is used to demonstrate building blocks of biomimetic learning via dynamical memristive circuits that implement biomimetic learning rules in a self-training neural network, with dynamical memristive weights that are capable of associative lifelong learning. Successful training of the dynamical memristive neural network to perform image classification of handwritten digits is shown, including lifelong learning by having the dynamical memristive network relearn different characters in succession. An analog computing architecture that learns to associate input-to-input correlations is also introduced, with examples demonstrating image classification and pattern recognition without convolution. The biomimetic functions shown in this paper result from fully ion-driven memristive circuits devoid of integrating capacitors and thus are instructive for exploiting the immense potential of memristive technology for neuromorphic computation in hardware and allowing a common architecture to be applied to a wide range of learning rules, including STDP, magnitude, frequency, and pulse shape among others, to enable an inorganic implementation of the complex heterogeneity of biological neural systems.

show abstract

SNN using color-opponent and attention mechanisms for object recognition

Yao,

Gao,

2025

Pattern Recognition

View full text Add to dashboard Cite

Early Termination Based Training Acceleration for an Energy-Efficient SNN Processor Design

Cited by 5 publications

References 36 publications

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

ADC/DAC-Free Analog Acceleration of Deep Neural Networks With Frequency Transformation

Dynamical memristive neural networks and associative self-learning architectures using biomimetic devices

SNN using color-opponent and attention mechanisms for object recognition

Contact Info

Product

Resources

About