Fundamental Limits on Energy-Delay-Accuracy of In-Memory Architectures in Inference Applications

Gonugondla, Sujan K.; Sakr, Charbel; Dbouk, Hassan; Shanbhag, Naresh R.

doi:10.1109/tcad.2021.3124757

Cited by 9 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such an investigation requires a detailed analysis of the IMC compute models and circuit methods employed by various reported designs. Though this study is beyond the scope of our paper, we point out preliminary work in this direction by us [104], [107], [108] and others [109].…”

Section: Summary and Discussionmentioning

confidence: 95%

“…1) Comprehending the fundamental efficiency vs. accuracy trade-offs in IMCs (see [107], [110]), including minimizing the column ADC's energy cost (see [97], [100], [104]). [circuits, statistical analysis] 2) Developing algorithmic approaches such as Shannoninspired statistical error compensation (SEC) [38], [111], [112] and approximate computing [47] to enhance the accuracy of IMCs beyond what is possible via purely 3) Leveraging emerging devices, e.g., MRAM, FeFET, to design IMCs with unique energy, latency, accuracy trade-offs.…”

Section: Summary and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

In-memory computing (IMC) architectures have emerged as a compelling platform to implement energy efficient machine learning (ML) systems. However, today, the energy efficiency gains provided by IMC designs seem to be leveling off and it is not clear what the limiting factors are. The conceptual complexity of IMCs combined with the absence of a rigorous benchmarking methodology makes it difficult to gauge progress and identify bottlenecks in this exciting field. This paper presents a benchmarking methodology for IMCs comprising: 1) a compositional view of IMCs that enables one to parse an IMC design into its canonical components; 2) a set of benchmarking metrics to quantify the performance, efficiency, and accuracy of IMCs; and 3) a strategy for analyzing the reported IMC data and metrics. We apply the proposed benchmarking methodology on an extensive database of IMC metrics extracted from > 70 IC designs published since 2018, in order to infer and comprehend trends in this area.Our benchmarking effort indicates: 1) SRAM-based IMCs show a clear win in terms of energy efficiency and compute density over digital accelerators at the bank-level but the energy efficiency gap reduces dramatically when comparing at the processor level; 2) eNVM-based IMCs lag behind SRAM-based IMCs in terms of both energy efficiency and compute density, and surprisingly lag digital accelerators in terms of compute density; 3) the compute (bank-level) accuracy of IMCs, though a critical metric, is pervasively neglected in publications as is the energy vs. accuracy trade-off inherent to IMCs.

show abstract

Section: Summary and Discussionmentioning

confidence: 95%

Section: Summary and Discussionmentioning

confidence: 99%

Benchmarking In-Memory Computing Architectures

Shanbhag

Roy

2022

IEEE Open J. Solid-State Circuits Soc.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The analog SNDR, SNDR a , factors out the impact of the ADC on the compute accuracy so as to provide a clear picture of the impact of array noise and variations on compute accuracy. Furthermore, since SNDR a ≥ SNDR d , SNDR a sets an upper bound on SNDR d [41]. Thus, the compute SNDR (SNDR a and SNDR d ) are a task-agnostic banklocalized metrics of accuracy that IMC designers can use to ensure that their IMC designs deliver the desired network level accuracy, e.g., classification accuracy.…”

Section: B Compute Sndrmentioning

confidence: 99%

“…It is quite well-known [41] that the compute SNDR of any IMC, either SRAM-or eNVM-based, trades-off with the dot-product dimension N . This trade-off occurs primarily because IMCs pack multiple output levels within a limited (current or voltage) swing range.…”

Section: B Sndra Vs Dot-product Dimension Nmentioning

confidence: 99%

Energy-Accuracy Trade-Offs for Resistive In-Memory Computing Architectures

Roy,

Shanbhag

2024

IEEE J. Explor. Solid-State Comput. Devices Circuits

Self Cite

View full text Add to dashboard Cite

Resistive in-memory computing (IMC) architectures currently lag behind SRAM IMCs and digital accelerators in both energy efficiency and compute density due to their low compute accuracy. This paper proposes the use of signal-to-noise-plus-distortion ratio (SNDR) to quantify the compute accuracy of IMCs and identify the device, circuit, and architectural parameters that affect it. We further analyze the fundamental limits on the SNDR of MRAM, ReRAM, and FeFET-based IMCs employing parameter variation and noise models that were validated against measured results from a recent MRAMbased IMC prototype in a 22 nm process. At high output signal magnitude, we find that the maximum achievable SNDR is limited by the pre-ADC array non-idealities such as the conductance variations, parasitic resistances, and current mirror mismatch, whereas the ADC thermal noise limits the SNDR at small signal magnitudes. Furthermore, for large dot-product (DP) dimensions (N > 50), the maximum achievable SNDR is highest for FeFET, followed by ReRAM, and then MRAM. Finally, the increase in conductance contrast (g on /g off ) enhances the maximum achievable SNDR only until it reaches a value of approximately 12. ReRAMs and FeFETs demonstrate high energy efficiencies while achieving high SNDR, as their low conductance values lead to lower currents and lower noise due to wire parasitics. In all cases, across all three device types, DP dimension, ADC precision, and conductance contrast, the maximum achievable SNDR is found to be in the range of 18 dB-to-22 dB, barely meeting the minimum needed for achieving an inference accuracy close to an equivalent fixed-point digital architecture. Finally, we demonstrate a network-level accuracy of 84.5 % when mapping a ResNet-20 (CIFAR-10) by ReRAMbased architecture at a SNDR of 22 dB, which MRAM and FeFET-based architectures cannot realize. This result clearly implies the need for other approaches, e.g., algorithmic and learning-based methods, to improve the inference accuracy of resistive IMC architectures.

show abstract

“…where the new notations along with their values are in Table 4. Following 65 nm CMOS technology limitations, we keep the array parameters similar to Kang et al (2018), and T adc and E adc for our 6-bit SNN are obtained by extending the circuit simulation results of Ali et al (2020) with the ADC energy and delay models proposed in Gonugondla et al (2020).…”

Section: Pim Hardwarementioning

confidence: 99%

ACE-SNN: Algorithm-Hardware Co-design of Energy-Efficient & Low-Latency Deep Spiking Neural Networks for 3D Image Recognition

et al. 2022

View full text Add to dashboard Cite

High-quality 3D image recognition is an important component of many vision and robotics systems. However, the accurate processing of these images requires the use of compute-expensive 3D Convolutional Neural Networks (CNNs). To address this challenge, we propose the use of Spiking Neural Networks (SNNs) that are generated from iso-architecture CNNs and trained with quantization-aware gradient descent to optimize their weights, membrane leak, and firing thresholds. During both training and inference, the analog pixel values of a 3D image are directly applied to the input layer of the SNN without the need to convert to a spike-train. This significantly reduces the training and inference latency and results in high degree of activation sparsity, which yields significant improvements in computational efficiency. However, this introduces energy-hungry digital multiplications in the first layer of our models, which we propose to mitigate using a processing-in-memory (PIM) architecture. To evaluate our proposal, we propose a 3D and a 3D/2D hybrid SNN-compatible convolutional architecture and choose hyperspectral imaging (HSI) as an application for 3D image recognition. We achieve overall test accuracy of 98.68, 99.50, and 97.95% with 5 time steps (inference latency) and 6-bit weight quantization on the Indian Pines, Pavia University, and Salinas Scene datasets, respectively. In particular, our models implemented using standard digital hardware achieved accuracies similar to state-of-the-art (SOTA) with ~560.6× and ~44.8× less average energy than an iso-architecture full-precision and 6-bit quantized CNN, respectively. Adopting the PIM architecture in the first layer, further improves the average energy, delay, and energy-delay-product (EDP) by 30, 7, and 38%, respectively.

show abstract

Fundamental Limits on Energy-Delay-Accuracy of In-Memory Architectures in Inference Applications

Cited by 9 publications

References 47 publications

Benchmarking In-Memory Computing Architectures

Benchmarking In-Memory Computing Architectures

Energy-Accuracy Trade-Offs for Resistive In-Memory Computing Architectures

ACE-SNN: Algorithm-Hardware Co-design of Energy-Efficient & Low-Latency Deep Spiking Neural Networks for 3D Image Recognition

Contact Info

Product

Resources

About