A 0.32–128 TOPS, Scalable Multi-Chip-Module-Based Deep Neural Network Inference Accelerator With Ground-Referenced Signaling in 16 nm

Zimmer, Brian; Venkatesan, Rangharajan; Shao, Yakun Sophia; Clemons, Jason; Fojtik, Matthew; Jiang, Nan; Keller, Ben; Klinefelter, Alicia; Pinckney, Nathaniel; Raina, Priyanka; Tell, Stephen G.; Zhang, Yanqing; Dally, William J.; Emer, Joel; Gray, C. Thomas; Keckler, Stephen W.; Khailany, Brucek

doi:10.1109/jssc.2019.2960488

Cited by 76 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, research on single-chip multicore processor architecture has gradually increased, with many students using it for some connections; for example, the new server group is a multistudent connection [1]. However, if the multicore processor has not yet reached the end of the process, the operating system will not be able to use the quality of the standard varieties of the process at all if it is always used directly in the process.…”

Section: Introductionmentioning

confidence: 99%

Low-Power Task Scheduling Algorithm for the Multicore Processor System Based on the Genetic Algorithm

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

In order to solve the periodic hard real-time tasks with dependencies on multicore processors, the author proposes a low-power task scheduling algorithm for multicore processor systems based on the genetic algorithm. This method first uses the RDAG algorithm to separate the tasks and then takes the lowest power consumption as the principle; a genetic algorithm is used to determine the task mapping. Experimental results show that based on the power consumption model of Intel PXA270, several random task sets are used for simulation experiments, which shows that this method saves 20% to 30% of the energy consumption compared with the existing methods. This method effectively shortens the completion time of tasks, improves the utilization efficiency of multicore system resources, improves the parallel computing capability of multicore systems, reduces the average response time of tasks, and improves the throughput and resource utilization of multicore systems.

show abstract

Section: Introductionmentioning

confidence: 99%

Low-Power Task Scheduling Algorithm for the Multicore Processor System Based on the Genetic Algorithm

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Therefore, conservatively, we expect the overall EDP to improve by at least 535 times when scaling the design from 130-nm to 7-nm technology. Extended Data Table 2 shows that such scaling will enable NeuRRAM to deliver higher energy and area efficiency than today's state-of-the-art edge inference accelerators [58][59][60][61] .…”

Section: Projection Of Neurram Energy Efficiency With Technology Scalingmentioning

confidence: 99%

A compute-in-memory chip based on resistive random-access memory

Wan

Kubendran

Schaefer

et al. 2022

Nature

Self Cite

399

181

View full text Add to dashboard Cite

Realizing increasingly complex artificial intelligence (AI) functionalities directly on edge devices calls for unprecedented energy efficiency of edge hardware. Compute-in-memory (CIM) based on resistive random-access memory (RRAM)1 promises to meet such demand by storing AI model weights in dense, analogue and non-volatile RRAM devices, and by performing AI computation directly within RRAM, thus eliminating power-hungry data movement between separate compute and memory2–5. Although recent studies have demonstrated in-memory matrix-vector multiplication on fully integrated RRAM-CIM hardware6–17, it remains a goal for a RRAM-CIM chip to simultaneously deliver high energy efficiency, versatility to support diverse models and software-comparable accuracy. Although efficiency, versatility and accuracy are all indispensable for broad adoption of the technology, the inter-related trade-offs among them cannot be addressed by isolated improvements on any single abstraction level of the design. Here, by co-optimizing across all hierarchies of the design from algorithms and architecture to circuits and devices, we present NeuRRAM—a RRAM-based CIM chip that simultaneously delivers versatility in reconfiguring CIM cores for diverse model architectures, energy efficiency that is two-times better than previous state-of-the-art RRAM-CIM chips across various computational bit-precisions, and inference accuracy comparable to software models quantized to four-bit weights across various AI tasks, including accuracy of 99.0 percent on MNIST18 and 85.7 percent on CIFAR-1019 image classification, 84.7-percent accuracy on Google speech command recognition20, and a 70-percent reduction in image-reconstruction error on a Bayesian image-recovery task.

show abstract

“…Each weight buffer bank has a read port that feeds into a floating-point vector MAC that provides scalability along a vector dimension of 16 (similar to the PE architecture in [52]).…”

Section: A Processing Elementmentioning

confidence: 99%

A 16-nm SoC for Noise-Robust Speech and NLP Edge AI Inference With Bayesian Sound Source Separation and Attention-Based DNNs

Tambe

Yang

et al. 2023

IEEE J. Solid-State Circuits

View full text Add to dashboard Cite

The proliferation of personal artificial intelligence (AI) -assistant technologies with speech-based conversational AI interfaces is driving the exponential growth in the consumer Internet of Things (IoT) market. As these technologies are being applied to keyword spotting (KWS), automatic speech recognition (ASR), natural language processing (NLP), and text-to-speech (TTS) applications, it is of paramount importance that they provide uncompromising performance for context learning in long sequences, which is a key benefit of the attention mechanism, and that they work seamlessly in polyphonic environments. In this work, we present a 25-mm 2 system-on-chip (SoC) in 16-nm FinFET technology, codenamed SM6, which executes end-to-end speech-enhancing attention-based ASR and NLP workloads. The SoC includes: 1) FlexASR, a highly reconfigurable NLP inference processor optimized for whole-model acceleration of bidirectional attention-based sequence-to-sequence (seq2seq) deep neural networks (DNNs); 2) a Markov random field source separation engine (MSSE), a probabilistic graphical model accelerator for unsupervised inference via Gibbs sampling, used for sound source separation; 3) a dual-core Arm Cortex A53 CPU cluster, which provides on-demand single Instruction/multiple data (SIMD) fast fourier transform (FFT) processing and performs various application logic (e.g., expectation-maximization (EM) algorithm and 8-bit floating-point (FP8) quantization); and 4) an always-ON M0 subsystem for audio detection and power management. Measurement results demonstrate the efficiency ranges of 2.6-7.8 TFLOPs/W and 4.33-17.6 Gsamples/s/W for FlexASR and MSSE, respectively; MSSE denoising performance allowing Manuscript

show abstract

A 0.32–128 TOPS, Scalable Multi-Chip-Module-Based Deep Neural Network Inference Accelerator With Ground-Referenced Signaling in 16 nm

Cited by 76 publications

References 17 publications

Low-Power Task Scheduling Algorithm for the Multicore Processor System Based on the Genetic Algorithm

Low-Power Task Scheduling Algorithm for the Multicore Processor System Based on the Genetic Algorithm

A compute-in-memory chip based on resistive random-access memory

A 16-nm SoC for Noise-Robust Speech and NLP Edge AI Inference With Bayesian Sound Source Separation and Attention-Based DNNs

Contact Info

Product

Resources

About