7.1 An 11.5TOPS/W 1024-MAC Butterfly Structure Dual-Core Sparsity-Aware Neural Processing Unit in 8nm Flagship Mobile SoC

Song, Jinouk; Cho, Yunkyo; Park, Jun‐Seok; Jang, Junwon; Leev, Sehwan; Song, Joonho; Lee, Jae-Gon; Kang, I-Seok

doi:10.1109/isscc.2019.8662476

Cited by 89 publications

(31 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The NPU of the Exynos 9820 supports only quantized inference and consists of the controller and two cores ( Fig. 2) having 1024 multiply-accumulate (MAC) units [78]. The NPU controller has a CPU, a direct memory access (DMA) unit, code SRAM and a network controller.…”

Section: Samsung Chipsets / Eden Sdkmentioning

confidence: 99%

“…When running the computations, the NPU can also skip weights that are zero to improve convolution efficiency. A much more detailed description of the Exynos NPU can be found in [78]. We strongly recommend reading this article for everyone interested in the general functioning of NPUs as it provides an excellent overview on all network / data processing stages and possible bottlenecks.…”

Section: Samsung Chipsets / Eden Sdkmentioning

confidence: 99%

See 1 more Smart Citation

AI Benchmark: All About Deep Learning on Smartphones in 2019

Ignatov

Timofte

Kulik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

198

106

View full text Add to dashboard Cite

The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs. The current 4th generation of mobile NPUs is already approaching the results of CUDAcompatible Nvidia graphics cards presented not long ago, which together with the increased capabilities of mobile deep learning frameworks makes it possible to run complex and deep AI models on mobile devices. In this paper, we evaluate the performance and compare the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference. We also discuss the recent changes in the Android ML pipeline and provide an overview of the deployment of deep learning models on mobile devices. All numerical results provided in this paper can be found and are regularly updated on the official project website 1 . * We also thank Oli Gaymond (ogaymond@google.com), Google Inc., for writing and editing section 3.1 of this paper. 1

show abstract

Section: Samsung Chipsets / Eden Sdkmentioning

confidence: 99%

Section: Samsung Chipsets / Eden Sdkmentioning

confidence: 99%

AI Benchmark: All About Deep Learning on Smartphones in 2019

Ignatov

Timofte

Kulik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

198

106

View full text Add to dashboard Cite

show abstract

“…In the same vein, to assess the impact of precision quantization across both the accuracy and robustness of the developed audio models, we investigated the performance of each model across four evaluation setups under different bitwidths in Figure 9(a) for = 0.001. For our experiments, we employed three reduced-precision representations using the quantization method discussed in §2.3.2: 16-bit (INT16) fixed-point which has been extensively studied to yield similar accuracy to the FP32 baseline [11,35], 8-bit (INT8) and 4-bit (INT4) fixed-point, which are suitable for hardware platforms with native support for low-precision arithmetic, such as the INT8-enabled mobile deep-learning accelerators by Qualcomm [49], Nvidia [50,51], Arm [52] and Samsung [53], and INT4 on Nvidia's Turing GPUs [54,55]. As shown in Figure 9(a), a bitwidth of 16 bits follows closely the behavior of the original, non-quantized models across all types of attacks.…”

Section: Transferability Of Adversarial Audio Examplesmentioning

confidence: 99%

Countering Acoustic Adversarial Attacks in Microphone-equipped Smart Home Devices

Bhattacharya

Manousakas

Ramos

et al. 2020

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

Deep neural networks (DNNs) continue to demonstrate superior generalization performance in an increasing range of applications, including speech recognition and image understanding. Recent innovations in compression algorithms, design of efficient architectures and hardware accelerators have prompted a rapid growth in deploying DNNs on mobile and IoT devices to redefine user experiences. Relying on the superior inference quality of DNNs, various voice-enabled devices have started to pervade our everyday lives and are increasingly used for, e.g., opening and closing doors, starting or stopping washing machines, ordering products online, and authenticating monetary transactions. As the popularity of these voice-enabled services increases, so does their risk of being attacked. Recently, DNNs have been shown to be extremely brittle under adversarial attacks and people with malicious intentions can potentially exploit this vulnerability to compromise DNN-based voice-enabled systems. Although some existing work already highlights the vulnerability of audio models, very little is known of the behaviour of compressed on-device audio models under adversarial attacks. This paper bridges this gap by investigating thoroughly the vulnerabilities of compressed audio DNNs and makes a stride towards making compressed models robust. In particular, we propose a stochastic compression technique that generates compressed models with greater robustness to adversarial attacks. We present an extensive set of evaluations on adversarial vulnerability and robustness of DNNs in two diverse audio recognition tasks, while considering two popular attack algorithms: FGSM and PGD. We found that error rates of conventionally trained audio DNNs under attack can be as high as 100%. Under both white-and black-box attacks, our proposed approach is found to decrease the error rate of DNNs under attack by a large margin.

show abstract

“…Neural Network (NN) Inference: Hardware accelerators for NN inference are rapidly becoming an integral part of system-on-chips (SoCs). They typically comprise large arrays of multiply-add (MADD) circuits [4] providing a considerable increase in inference speed. In order to meet tighter and tighter throughput constraints, state of the art traditionally trades-off inference speed with accuracy in which the precision of MADD circuits is reduced (i.e., achieve higher frequency) at the cost of some accuracy loss [4], [5].…”

Section: Introductionmentioning

confidence: 99%

“…They typically comprise large arrays of multiply-add (MADD) circuits [4] providing a considerable increase in inference speed. In order to meet tighter and tighter throughput constraints, state of the art traditionally trades-off inference speed with accuracy in which the precision of MADD circuits is reduced (i.e., achieve higher frequency) at the cost of some accuracy loss [4], [5]. However, to satisfy such tight throughput constraints, NN accelerators integrate thousands of MADD units [4] resulting in a significant increase in energy consumption, which might not be tolerated.…”

Section: Introductionmentioning

confidence: 99%

Impact of NCFET on Neural Network Accelerators

et al. 2021

View full text Add to dashboard Cite

This is the first work to investigate the impact that Negative Capacitance Field-Effect Transistor (NCFET) brings on the efficiency and accuracy of future Neural Networks (NN). NCFET is at the forefront of emerging technologies, especially after it has become compatible with the existing fabrication process of CMOS. Neural Network inference accelerators are becoming ubiquitous in modern SoCs and there is an ever-increasing demand for tighter and tighter throughput constraints and lower energy consumption. To explore the benefits that NCFET brings to NN inference regarding frequency, energy, and accuracy, we investigate different configurations of the multiply-add (MADD) circuit, which is the core computational unit in any NN accelerator. We demonstrate that, compared to the baseline 7nm FinFET technology, its negative capacitance counterpart reduces the energy by 55%, without any frequency reduction. In addition, it enables leveraging higher computational precision, which results to a considerable improvement in the inference accuracy. Importantly, the achieved accuracy improvement comes also together with a significant energy reduction and without any loss in frequency.

show abstract

7.1 An 11.5TOPS/W 1024-MAC Butterfly Structure Dual-Core Sparsity-Aware Neural Processing Unit in 8nm Flagship Mobile SoC

Cited by 89 publications

References 4 publications

AI Benchmark: All About Deep Learning on Smartphones in 2019

AI Benchmark: All About Deep Learning on Smartphones in 2019

Countering Acoustic Adversarial Attacks in Microphone-equipped Smart Home Devices

Impact of NCFET on Neural Network Accelerators

Contact Info

Product

Resources

About