AI Benchmark: Running Deep Neural Networks on Android Smartphones

Ignatov, Andrey; Timofte, Radu; Chou, William; Wang, Ke; Wu, Max; Hartley, Tim; Gool, Luc Van

doi:10.1007/978-3-030-11021-5_19

Cited by 239 publications

(199 citation statements)

References 48 publications

Supporting

Mentioning

198

Contrasting

Unclassified

Order By: Relevance

“…Though some TensorFlow Lite issues mentioned last year [31] were solved in its current releases, we still recommend using it with great precaution. For instance, in its latest official build (1.14), the interaction with NNAPI was completely broken, leading to enormous losses and random outputs during the first two inferences.…”

Section: Deep Learning On Smartphonesmentioning

confidence: 99%

“…As to other MediaTek chipsets presented this year, the Helio G90 and the Helio P65 are also providing hardware acceleration for float and quantized AI models. The former uses a separate APU (1st gen.) with a similar architecture as the one in the Helio P60 / P70 chipsets [31]. The Helio P65 does not have a dedicated APU module and is running all models on a Mali-G52 MP2 GPU.…”

Section: Mediatek Chipsets / Neuropilot Sdkmentioning

confidence: 99%

“…In total, 12 public versions of the AI Benchmark application were released since the beginning of the project. The second generation (2.0.0) was described in detail in the preceding paper [31]. Below we briefly summarize the key changes introduced in the subsequent benchmark releases: -2.1.0 (release date: 13.10.2018) -this version brought a number of major changes to AI Benchmark.…”

Section: Ai Benchmark 30mentioning

confidence: 99%

“…Required Android 4.1 and above. Since a detailed technical description of AI Benchmark 2.0 was provided in [31], we here mainly focus on the updates and changes introduced by the latest release.…”

Section: Ai Benchmark 30mentioning

confidence: 99%

See 3 more Smart Citations

AI Benchmark: All About Deep Learning on Smartphones in 2019

Ignatov

Timofte

Kulik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

198

106

View full text Add to dashboard Cite

The performance of mobile AI accelerators has been evolving rapidly in the past two years, nearly doubling with each new generation of SoCs. The current 4th generation of mobile NPUs is already approaching the results of CUDAcompatible Nvidia graphics cards presented not long ago, which together with the increased capabilities of mobile deep learning frameworks makes it possible to run complex and deep AI models on mobile devices. In this paper, we evaluate the performance and compare the results of all chipsets from Qualcomm, HiSilicon, Samsung, MediaTek and Unisoc that are providing hardware acceleration for AI inference. We also discuss the recent changes in the Android ML pipeline and provide an overview of the deployment of deep learning models on mobile devices. All numerical results provided in this paper can be found and are regularly updated on the official project website 1 . * We also thank Oli Gaymond (ogaymond@google.com), Google Inc., for writing and editing section 3.1 of this paper. 1

show abstract

Section: Deep Learning On Smartphonesmentioning

confidence: 99%

Section: Mediatek Chipsets / Neuropilot Sdkmentioning

confidence: 99%

Section: Ai Benchmark 30mentioning

confidence: 99%

Section: Ai Benchmark 30mentioning

confidence: 99%

See 2 more Smart Citations

AI Benchmark: All About Deep Learning on Smartphones in 2019

Ignatov

Timofte

Kulik

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

198

106

View full text Add to dashboard Cite

show abstract

“…The incorporation of specific platform constraints to such approaches involves modeling how the network architecture relates with the optimization target. As a first step for modeling the performance of embedded CNNs, recent studies have carried out systematic benchmarking on several hardware systems [6,[27][28][29]. Gaining in specificity, an energy estimation methodology for CNN accelerators has been introduced in [30,31].…”

Section: Related Workmentioning

confidence: 99%

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices

Velasco-Montero

Fernández-Berni

Carmona-Galán

et al. 2020

IEEE Internet Things J.

View full text Add to dashboard Cite

This paper presents PreVIous, a methodology to predict the performance of convolutional neural networks (CNNs) in terms of throughput and energy consumption on vision-enabled devices for the Internet of Things. CNNs typically constitute a massive computational load for such devices, which are characterized by scarce hardware resources to be shared among multiple concurrent tasks. Therefore, it is critical to select the optimal CNN architecture for a particular hardware platform according to prescribed application requirements. However, the zoo of CNN models is already vast and rapidly growing. To facilitate a suitable selection, we introduce a prediction framework that allows to evaluate the performance of CNNs prior to their actual implementation. The proposed methodology is based on PreVIousNet, a neural network specifically designed to build accurate per-layer performance predictive models. PreVIousNet incorporates the most usual parameters found in state-of-the-art network architectures. The resulting predictive models for inference time and energy have been tested against comprehensive characterizations of seven well-known CNN models running on two different software frameworks and two different embedded platforms. To the best of our knowledge, this is the most extensive study in the literature concerning CNN performance prediction on low-power low-cost devices. The average deviation between predictions and real measurements is remarkably low, ranging from 3% to 10%. This means state-of-the-art modeling accuracy. As an additional asset, the fine-grained a priori analysis provided by PreVIous could also be exploited by neural architecture search engines.

show abstract

Efficient and Explainable Deep Neural Networks for Airway Symptom Detection in Support of Wearable Health Technology

Groh

Lei

Martignetti

et al. 2022

Advanced Intelligent Systems

View full text Add to dashboard Cite

Mobile health wearables are often embedded with small processors for signal acquisition and analysis. These embedded wearable systems are, however, limited with low available memory and computational power. Advances in machine learning, especially deep neural networks (DNNs), have been adopted for efficient and intelligent applications to overcome constrained computational environments. Herein, evolutionary algorithms are used to find novel DNNs that are accurate in classifying airway symptoms while allowing wearable deployment. As opposed to typical microphone‐acoustic signals, mechano‐acoustic data signals, which did not contain identifiable speech information for better privacy protection, are acquired from laboratory‐generated and publicly available datasets. The optimized DNNs had a low model file size of less than 150 kB and predicted airway symptoms of interest with 81.49% accuracy on unseen data. By performing explainable AI techniques, namely occlusion experiments and class activation maps, mel‐frequency bands up to 8,000 Hz are found as the most important feature for the classification. It is further found that DNN decisions are consistently relying on these specific features, fostering trust and transparency of the proposed DNNs. The proposed efficient and explainable DNN is expected to support edge computing on mechano‐acoustic sensing wearables for remote, long‐term monitoring of airway symptoms.

show abstract

AI Benchmark: Running Deep Neural Networks on Android Smartphones

Cited by 239 publications

References 48 publications

AI Benchmark: All About Deep Learning on Smartphones in 2019

AI Benchmark: All About Deep Learning on Smartphones in 2019

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices

Efficient and Explainable Deep Neural Networks for Airway Symptom Detection in Support of Wearable Health Technology

Contact Info

Product

Resources

About