A 3.43TOPS/W 48.9pJ/pixel 50.1nJ/classification 512 analog neuron sparse coding neural network with on-chip learning and classification in 40nm CMOS

Buhler, Fred; Brown, Peter; Li, Jiabo; Chen, Thomas; Zhang, Zhengya; Flynn, Michael P.

doi:10.23919/vlsic.2017.8008536

Cited by 52 publications

(25 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work, spiking hardware architectures are only analyzed in their ability to support DNNs. Dedicated spiking hardware architectures designed for specific DNNs [45], [46] achieve comparable AT E efficiencies as their non-spiking counterparts, but their classification accuracy is 5 % or more below expectations for the given benchmark. However, it is also possible for spiking hardware accelerators to achieve competitive accuracies, e.g.…”

Section: ) Spiking Neural Networkmentioning

confidence: 99%

“…A comparison between both design styles shows that MS designs are more common [26]- [29], [39], [45] than purely analog ones [31], [40]. Both target mostly small benchmarks, like MNIST and CIFAR 10 (cf.…”

Section: ) Analog and Ms Design Stylementioning

confidence: 99%

“…Content may change prior to final publication. [25], [30], [32]- [38], [42], [46], [47], [53], [55], [57], [ [29], [31], [39], [40], [45] → → → ↑ Compression I [49], [51], [53], [55], [57], [ [25], [36]- [38], [48], [49], [51], [57] ↓ ↑ ↓ → Zero-Skipping I [49], [51], [53], [55], [57], [58] ↓ ↑ ↓ → Bit-Serial Processing I [24], [25], [34], [36], [37]…”

Section: ) Voltage Underscalingmentioning

confidence: 99%

See 2 more Smart Citations

Efficiency Versus Accuracy: A Review of Design Techniques for DNN Hardware Accelerators

Latotzke

Gemmeke

2021

IEEE Access

View full text Add to dashboard Cite

Deep neural networks (DNNs) have surpassed other algorithms in analyzing today's abundant data. Due to the security and delay requirements of the given applications, analytical data extraction should happen on edge devices. Edge devices, however, struggle to support the increasingly complex DNNs because of the models' high computational load and number of employed parameters. Thus, edge devices need support by efficient accelerators to process DNNs. However, the design of DNN accelerators remains challenging, as there is a lack of established design techniques directed towards a specific design point in terms of energy budget, area, time-to-solution, and classification accuracy. This paper fills this gap by providing a quantitative, large-scale, state-of-the-art comparison of DNN accelerators building on published data subjected to technology scaling and benchmark normalization. The leveled comparison stimulates learning from previous designs by considering the impact of each technique on energy, area, and timeto-solution. Furthermore, the key design techniques used in the DNN accelerators are classified according to their influence on the classification accuracy. Finally, we provide a discussion of hardware accelerators to support future designers in considering the trade-off between efficiency and accuracy in order to identify the most suitable techniques for certain benchmarks.

show abstract

Section: ) Spiking Neural Networkmentioning

confidence: 99%

Section: ) Analog and Ms Design Stylementioning

confidence: 99%

Section: ) Voltage Underscalingmentioning

confidence: 99%

See 1 more Smart Citation

Efficiency Versus Accuracy: A Review of Design Techniques for DNN Hardware Accelerators

Latotzke

Gemmeke

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…An analysis of the energy, area and accuracy tradeoffs is shown in Fig. 14 Kim et al [44], from Buhler et al [45] and TrueNorth, which was benchmarked on MNIST in [46]. In order to carry out comparison in a one-to-one basis, all area and energy numbers have been normalized to a 65-nm technology node.…”

Section: Tradeoff Analysis Of Energy Area and Accuracymentioning

confidence: 99%

MorphIC: A 65-nm 738k-Synapse/mm$^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning

Frenkel

Legat

Bol

2019

IEEE Trans. Biomed. Circuits Syst.

142

View full text Add to dashboard Cite

Recent trends in the field of neural network accelerators investigate weight quantization as a means to increase the resource-and power-efficiency of hardware devices. As full on-chip weight storage is necessary to avoid the high energy cost of off-chip memory accesses, memory reduction requirements for weight storage pushed toward the use of binary weights, which were demonstrated to have a limited accuracy reduction on many applications when quantization-aware training techniques are used. In parallel, spiking neural network (SNN) architectures are explored to further reduce power when processing sparse eventbased data streams, while on-chip spike-based online learning appears as a key feature for applications constrained in power and resources during the training phase. However, designing power-and area-efficient spiking neural networks still requires the development of specific techniques in order to leverage onchip online learning on binary weights without compromising the synapse density. In this work, we demonstrate MorphIC, a quadcore binary-weight digital neuromorphic processor embedding a stochastic version of the spike-driven synaptic plasticity (S-SDSP) learning rule and a hierarchical routing fabric for large-scale chip interconnection. The MorphIC SNN processor embeds a total of 2k leaky integrate-and-fire (LIF) neurons and more than two million plastic synapses for an active silicon area of 2.86mm 2 in 65nm CMOS, achieving a high density of 738k synapses/mm 2 . MorphIC demonstrates an order-of-magnitude improvement in the area-accuracy tradeoff on the MNIST classification task compared to previously-proposed SNNs, while having no penalty in the energy-accuracy tradeoff.

show abstract

“…1). However, top-down designs currently appear to be either spiking neural networks (SNNs) with event-driven processing at the expense of accuracy [16], [17] or binary neural networks (BNNs) This work was supported by the fonds européen de développement régional FEDER, the Wallonia within the "Wallonie-2020.EU" program, the Plan Marshall and the FRS-FNRS of Belgium. E-mail: charlotte.frenkel@uclouvain.be…”

Section: Introductionmentioning

confidence: 99%

A 28-nm Convolutional Neuromorphic Processor Enabling Online Learning with Spike-Based Retinas

Frenkel¹,

Legat²,

Bol³

2020

Preprint

View full text Add to dashboard Cite

In an attempt to follow biological information representation and organization principles, the field of neuromorphic engineering is usually approached bottom-up, from the biophysical models to large-scale integration in silico. While ideal as experimentation platforms for cognitive computing and neuroscience, bottom-up neuromorphic processors have yet to demonstrate an efficiency advantage compared to specialized neural network accelerators for real-world problems. Top-down approaches aim at answering this difficulty by (i) starting from the applicative problem and (ii) investigating how to make the associated algorithms hardware-efficient and biologically-plausible. In order to leverage the data sparsity of spike-based neuromorphic retinas for adaptive edge computing and vision applications, we follow a top-down approach and propose SPOON, a 28-nm event-driven CNN (eCNN). It embeds online learning with only 16.8-% power and 11.8-% area overheads with the biologically-plausible direct random target projection (DRTP) algorithm. With an energy per classification of 313nJ at 0.6V and a 0.32-mm 2 area for accuracies of 95.3% (on-chip training) and 97.5% (off-chip training) on MNIST, we demonstrate that SPOON reaches the efficiency of conventional machine learning accelerators while embedding onchip learning and being compatible with event-based sensors, a point that we further emphasize with N-MNIST benchmarking.

show abstract

A 3.43TOPS/W 48.9pJ/pixel 50.1nJ/classification 512 analog neuron sparse coding neural network with on-chip learning and classification in 40nm CMOS

Cited by 52 publications

References 0 publications

Efficiency Versus Accuracy: A Review of Design Techniques for DNN Hardware Accelerators

Efficiency Versus Accuracy: A Review of Design Techniques for DNN Hardware Accelerators

MorphIC: A 65-nm 738k-Synapse/mm$^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning

A 28-nm Convolutional Neuromorphic Processor Enabling Online Learning with Spike-Based Retinas

Contact Info

Product

Resources

About