On the Accuracy of Analog Neural Network Inference Accelerators

Xiao, T. Patrick; Feinberg, Benjamin A.; Bennett, Christopher H.; Prabhakar, V.; Saxena, Prashant Sahai; Agrawal, Vikesh; Agarwal, Sapan; Marinella, Matthew

doi:10.1109/mcas.2022.3214409

Cited by 24 publications

(12 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the contrary, if the focus is set on the sampling frequency (with reading times in the order of 10 ns), low-resolution/high-speed-flash ADC (Fig. 11e ) can be applied via time multiplexing to minimize die area as for instance ADCs with at least 8-bit resolution are necessary to achieve high (>90%) classification accuracy in a ResNET50-1.5 ANN used to classify the ImageNET 157 database or in a multi-layer perceptron to classify the breast cancer screening database 57 . This approach requires the use of analogue multiplexers (block 11).…”

Section: Structure Of Memristor-based Annsmentioning

confidence: 99%

Hardware implementation of memristor-based artificial neural networks

Aguirre,

Sebastian,

Le Gallo

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Artificial Intelligence (AI) is currently experiencing a bloom driven by deep learning (DL) techniques, which rely on networks of connected simple computing units operating in parallel. The low communication bandwidth between memory and processing units in conventional von Neumann machines does not support the requirements of emerging applications that rely extensively on large sets of data. More recent computing paradigms, such as high parallelization and near-memory computing, help alleviate the data communication bottleneck to some extent, but paradigm- shifting concepts are required. Memristors, a novel beyond-complementary metal-oxide-semiconductor (CMOS) technology, are a promising choice for memory devices due to their unique intrinsic device-level properties, enabling both storing and computing with a small, massively-parallel footprint at low power. Theoretically, this directly translates to a major boost in energy efficiency and computational throughput, but various practical challenges remain. In this work we review the latest efforts for achieving hardware-based memristive artificial neural networks (ANNs), describing with detail the working principia of each block and the different design alternatives with their own advantages and disadvantages, as well as the tools required for accurate estimation of performance metrics. Ultimately, we aim to provide a comprehensive protocol of the materials and methods involved in memristive neural networks to those aiming to start working in this field and the experts looking for a holistic approach.

show abstract

Section: Structure Of Memristor-based Annsmentioning

confidence: 99%

Hardware implementation of memristor-based artificial neural networks

Aguirre,

Sebastian,

Le Gallo

et al. 2024

Nat Commun

View full text Add to dashboard Cite

show abstract

“…Despite this programming error, it is possible to obtain high inference accuracy even on modern deep CNNs that use AIMC circuits thanks to the averaging over many values in each column. The accuracy possible for a particular CNN and dataset (e.g., ImageNet running on ResNet-50) depends on the details of the device being used to represent the weight and the error at a given conductance (Xiao et al, 2021b(Xiao et al, , 2022b. Resistive random access memory (ReRAM) (Marinella et al, 2018), phase-change memory (PCM) (Burr et al, 2010;Raoux et al, 2008), and SONOS (Xiao et al, 2022b) are the most mature in-production nonvolatile memory devices being considered as the tunable resistor in Figure 17(b).…”

Section: Devicesmentioning

confidence: 99%

“…The programming error of each of these devices is typically characterized as a function of conductance, which is plotted in Figure 18 for ReRAM, SO-NOS, and PCM. The SONOS (green curve) has the lowest error at low conductance, and this is ideal behavior and provides the highest CNN inference accuracy (Xiao et al, 2021b(Xiao et al, , 2022b. A drawback of SONOS is the slow drift of the state following initial programming, but this can be significantly accelerated under modest ionizing radiation (Xiao et al, 2021a).…”

Section: Devicesmentioning

confidence: 99%

Section: Devicesmentioning

confidence: 99%

See 1 more Smart Citation

Abisko: Deep codesign of an architecture for spiking neural networks using novel neuromorphic materials

Vetter

Date

Fahim

et al. 2023

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

The Abisko project aims to develop an energy-efficient spiking neural network (SNN) computing architecture and software system capable of autonomous learning and operation. The SNN architecture explores novel neuromorphic devices that are based on resistive-switching materials, such as memristors and electrochemical RAM. Equally important, Abisko uses a deep codesign approach to pursue this goal by engaging experts from across the entire range of disciplines: materials, devices and circuits, architectures and integration, software, and algorithms. The key objectives of our Abisko project are threefold. First, we are designing an energy-optimized high-performance neuromorphic accelerator based on SNNs. This architecture is being designed as a chiplet that can be deployed in contemporary computer architectures and we are investigating novel neuromorphic materials to improve its design. Second, we are concurrently developing a productive software stack for the neuromorphic accelerator that will also be portable to other architectures, such as field-programmable gate arrays and GPUs. Third, we are creating a new deep codesign methodology and framework for developing clear interfaces, requirements, and metrics between each level of abstraction to enable the system design to be explored and implemented interchangeably with execution, measurement, a model, or simulation. As a motivating application for this codesign effort, we target the use of SNNs for an analog event detector for a high-energy physics sensor.

show abstract

“…The requirements for ANN and SNN devices depend on their intended use, e.g., CMOS-compatible devices for state-of-the-art electronics and biocompatible devices for use in health. Many devices have been proposed for building these systems; − however, only a subset meets the performance requirements, − few are biocompatible, − and fewer leverage advanced biological behavior such as that of dendrites. − In biological systems, dendrites branch out from the neuronal body and process incoming spikes into nonspiking spatiotemporal signals. For example, the leaky integrate-and-fire model of neuronal behavior is composed of one leaky recurrent dendrite and a soma for an activation threshold.…”

mentioning

confidence: 99%

Graphene-Based Artificial Dendrites for Bio-Inspired Learning in Spiking Neuromorphic Systems

Liu,

Akinwande,

Kireev

et al. 2024

Nano Lett.

View full text Add to dashboard Cite

Analog neuromorphic computing systems emulate the parallelism and connectivity of the human brain, promising greater expressivity and energy efficiency compared to those of digital systems. Though many devices have emerged as candidates for artificial neurons and artificial synapses, there have been few device candidates for artificial dendrites. In this work, we report on biocompatible graphene-based artificial dendrites (GrADs) that can implement dendritic processing. By using a dual side-gate configuration, current applied through a Nafion membrane can be used to control device conductance across a trilayer graphene channel, showing spatiotemporal responses of leaky recurrent, alpha, and Gaussian dendritic potentials. The devices can be variably connected to enable higher-order neuronal responses, and we show through data-driven spiking neural network simulations that spiking activity is reduced by ≤15% without accuracy loss while low-frequency operation is stabilized. This positions the GrADs as strong candidates for energy efficient bio-interfaced spiking neural networks.

show abstract

On the Accuracy of Analog Neural Network Inference Accelerators

Cited by 24 publications

References 54 publications

Hardware implementation of memristor-based artificial neural networks

Hardware implementation of memristor-based artificial neural networks

Abisko: Deep codesign of an architecture for spiking neural networks using novel neuromorphic materials

Graphene-Based Artificial Dendrites for Bio-Inspired Learning in Spiking Neuromorphic Systems

Contact Info

Product

Resources

About