Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

Abstract

<jats:p>With the recent advances in both machine learning and embedded systems research, the demand to deploy computational models for real-time execution on edge devices has increased substantially. Without deploying computational models on edge devices, the frequent transmission of sensor data to the cloud results in rapid battery draining due to the energy consumption of wireless data transmission. This rapid power dissipation leads to a considerable reduction in the battery lifetime of the system, therefore jeopardizing the real-world utility of smart devices. It is well-established that for difficult machine learning tasks, models with higher performance often require more computation power and thus are not power-efficient choices for deployment on edge devices. However, the trade-offs between performance and power consumption are not well studied. While numerous methods (e.g., model compression) have been developed to obtain an optimal model, these methods focus on improving the efficiency of a "single" model. In an entirely new direction, we introduce an effective method to find a combination of "multiple" models that are optimal in terms of power-efficiency and performance by solving an optimization problem in which both performance and power consumption are taken into account. Experimental results demonstrate that on the ImageNet dataset, we can achieve a 20% energy reduction with only 0.3% accuracy drop compared to Squeeze-and-Excitation Networks. Compared to a pruned convolutional neural network for human activity recognition, while consuming 1.7% less energy, our proposed policy achieves 1.3% higher accuracy.</jats:p>

Hardware-Aware Affordance Detection for Application in Portable Embedded Systems

“…The deployment was performed on several hardware solutions and pros and cons were analyzed. This study paves the way to the setup of deployment strategies where given a set of constraints it is possible to retrieve the best solution by performing an optimization procedure [49]. In fact, such strategies may represent a critical step when focusing on specific applications of the proposed system.…”

MultiTASC++: A continuously adaptive scheduler for edge-based multi-device cascade inference

“…[6] investigates the use of multiple DNN models going beyond two element cascades, coupled with the evaluation of various decision metrics. Additionally, solutions have been proposed to deploy cascades under tight energy constraints [7,8]. Most of the aforementioned work focus on the image classi ication task and don't consider the problem of dynamic adaptation at runtime.…”

“…This new strategy involves placing the server within or in close proximity to the consumer environment, of-ten in the form of a dedicated AI hub designed to assist nearby devices [2]. Within this framework, cascade architectures have emerged as a notable deployment approach [4,5,6,7,8,9]. These architectures capitalize on the inherent variability in sample dif iculty, opting to process only the more challenging cases with a robust server-based model, while delegating the processing of simpler samples, which typically constitute the majority of the data stream, to on-device execution using a lightweight model.…”

Adaptive Energy Management for Self-Sustainable Wearables in Mobile Health

“…Specifically, we collect activity data with different sampling rates and design a classifier for each sampling rate. These datapoints are then used to obtain the relationship between sampling rate and accuracy (Mirzadeh and Ghasemzadeh 2020;Bhat et al 2019). After getting the above inputs, the algorithm first initializes the battery charging indicator to zero for intervals where the user performs critical activities.…”