ApproxTuner

Sharif, Hashim; Zhao, Yifan; Kotsifakou, Maria; Kothari, Akash; Schreiber, Ben Z.; Wang, Elizabeth; Sarita, Yasmin; Zhao, Nathan; Joshi, Keyur D.; Adve, Vikram; Misailovíc, Saša; Adve, Sarita V.

doi:10.1145/3437801.3446108

Cited by 16 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ApproxTuner [15] delivers heuristic-based search of the space of possible approximations of each individual network layer, so that a comprehensive speedup-inference accuracy trade-off curve is charted and the list of the most promising sets of approximations is identified. Yet, ApproxTuner does not take into account the peculiarities of the mobile platform and the predicted trade-off curves it draws do not reflect the actual performance observed on the mobiles.…”

Section: Mobiprox Builds Upon the Existing Work On Heterogeneous And ...mentioning

confidence: 99%

See 1 more Smart Citation

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Fabjančič,

Machidon,

Sharif

et al. 2024

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

Runtime-tunable context-dependent network compression would make mobile deep learning (DL) adaptable to often varying resource availability, input "difficulty", or user needs. The existing compression techniques significantly reduce the memory, processing, and energy tax of DL, yet, the resulting models tend to be permanently impaired, sacrificing the inference power for reduced resource usage. The existing tunable compression approaches, on the other hand, require expensive re-training, do not support arbitrary strategies for adapting the compression and do not provide mobile-ready implementations.In this paper we present Mobiprox, a framework enabling mobile DL with flexible precision. Mobiprox implements tunable approximations of tensor operations and enables runtimeadaptable approximation of the individual network layers. A profiler and a tuner included with Mobiprox identify the most promising neural network approximation configurations leading to the desired inference quality with the minimal use of resources. Furthermore, we develop control strategies that depending on contextual factors, such as the input data difficulty, dynamically adjust the approximation levels across a mobile DL model's layers. We implement Mobiprox in Android OS and through experiments in diverse mobile domains, including human activity recognition and spoken keyword detection, demonstrate that it can save up to 15% system-wide energy with a minimal impact on the inference accuracy.

show abstract

Section: Mobiprox Builds Upon the Existing Work On Heterogeneous And ...mentioning

confidence: 99%

“…However, ApproxHPVM targets server environments, generates only CUDA-ready binaries, and does not support compilation for mobile hardware (Android or iOS). With the help of Approx-Tuner [15], approximation levels within ApproxHPVM can be dynamically adapted, yet, the provided adaptation method is simple, reactive and context-oblivious.…”

Section: Introductionmentioning

confidence: 99%

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Fabjančič,

Machidon,

Sharif

et al. 2024

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

show abstract

“…ApproxTuner [40] adds support for heuristic-based search of the space of possible approximations of each individual network layer, so that a comprehensive speedup-inference accuracy trade-off curve is charted and the list of the most promising sets of approximations is identified.…”

Section: Preliminariesmentioning

confidence: 99%

“…for its deep learning models. To achieve this, for a given pre-trained neural network, Mobiprox, relying on ApproxTuner [40], first investigates the effect of different per NN-operation approximations on the inference accuracy and identifies speedup-inference accuracy trade-off points. Each of these points is realized through a different set of approximations (and is termed: configuration) applied to layers of a neural network.…”

Section: Mobiprox Framework 41 Overviewmentioning

confidence: 99%

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Fabjančič¹,

Machidon²,

Sharif³

et al. 2023

Preprint

View full text Add to dashboard Cite

Runtime-tunable context-dependent network compression would make mobile deep learning adaptable to often varying resource availability, input "difficulty", or user needs. The existing compression techniques significantly reduce the memory, processing, and energy tax of deep learning, yet, the resulting models tend to be permanently impaired, sacrificing the inference power for reduced resource usage. The existing tunable compression approaches, on the other hand, require expensive re-training, seldom provide mobile-ready implementations, and do not support arbitrary strategies for adapting the compression.In this paper we present Mobiprox, a framework enabling flexible-accuracy on-device deep learning. Mobiprox implements tunable approximations of tensor operations and enables runtime adaptation of individual network layers. A profiler and a tuner included with Mobiprox identify the most promising neural network approximation configurations leading to the desired inference quality with the minimal use of resources. Furthermore, we develop control strategies that depending on contextual factors, such as the input data difficulty, dynamically adjust the approximation level of a model. We implement Mobiprox in Android OS and through experiments in diverse mobile domains, including human activity recognition and spoken keyword detection, demonstrate that it can save up to 15% system-wide energy with a minimal impact on the inference accuracy.CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing; • Computing methodologies → Neural networks.

show abstract

Approximate execution and grouping of critical sections for performance‐accuracy tradeoff

Altuntaş

Arslan

Boz

2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryApproximate computing enhances the performance and energy efficiency of applications, while still achieving acceptable accuracy. Some of the multithreaded applications can tolerate the accuracy loss when critical sections are approximately executed, which in turn will eliminate the synchronization overhead of these applications and increase their performance. In this study, our objective is to explore the behavior of the critical sections and selectively skip the ones yielding performance improvements with an acceptable accuracy loss. We have observed the behavior of 62 critical sections of 4 selected applications. We have grouped them depending on their effects on program execution and skipped or approximated them accordingly. Our experimental study indicates that skipping 76% of the critical sections offers 2.5 performance gain with 16% accuracy loss for Raytrace whereas 1.4 performance improvement with 17% accuracy loss is obtained for Radiosity on average when 36% of the critical sections are skipped. For Water_NSquared the performance gain is 1.1 with 9% accuracy loss on average with 79% of critical sections skipped. For the Ocean_CP application we see a performance improvement of 1.6 with accuracy loss 1% when we skip 38% of critical sections.

show abstract

ApproxTuner

Cited by 16 publications

References 44 publications

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Mobiprox: Supporting Dynamic Approximate Computing on Mobiles

Approximate execution and grouping of critical sections for performance‐accuracy tradeoff

Contact Info

Product

Resources

About