Energy Optimal Activation of Processors for the Execution of a Single Task with Unknown Size

Abstract

A key objective in the management of modern computer systems consists in minimizing the electrical energy consumed by processing resources while satisfying certain target performance criteria. In this paper, we consider the execution of a single task with unknown size on top of a service system that offers a limited number of processing speeds, say N , and investigate the problem of finding a speed profile that minimizes the resulting energy consumption subject to a deadline constraint. Existing works mainly investigated this problem when speed profiles are continuous functions. In contrast, the novelty of our work is to consider discontinuous speed profiles, i.e., a case that arises naturally when the underlying computational platform offers a finite number of speeds.In our main result, we show that the computation of an optimal speed profile boils down to solving a convex optimization problem. Under mild assumptions, for such convex optimization we prove some structural results that yield the formulation of an extremely efficient solution algorithm. Specifically, we show that the optimal speed profile can be computed by solving O(log N ) one-dimensional equations. Our results hold when the task size follows a known probability distribution function and the set of available speeds, if listed in increasing order, forms a sublinear concave sequence.

A Stochastic Approach for Scheduling AI Training Jobs in GPU-Based Systems

“…The analysis in [11] of the performance achieved on real clusters demonstrated that the difference between the total costs (based on jobs performance) obtained by our models and the measured costs on the real system is below 13% (significantly lower than the savings, in 32-80% range, we achieved with respect to other methods as it will be demonstrated in Section VII). Finally, the paper in [17], which serves as the foundation for the development of our stochastic scheduler, is included in Section V for comprehensive coverage. This paper presents an analytical model for the ideal scheduling of individual Deep Learning jobs within a system based on GPUs.…”

“…The model presented in this section was first introduced in a previous work [17]. In Section VI, we will rely on this model to solve the global optimization problem heuristically by using the optimal GPU allocation for each job in isolation and then by constructing a global schedule by superposing each individual schedule.…”

“…Furthermore, this work extends [17], which was focused on determining the optimal schedule for a single job in a context where resource requirements can always be satisfied, using it as a starting point to determine good-quality heuristic solutions for the whole RS-JS problem in a more realistic setting.…”

“…Therefore, we develop a heuristic method, called STochastic Scheduler (STS). Based on a stochastic model considering a single DL job, initially proposed in [17], STS determines a dynamic schedule by minimizing the expected value of energy costs while meeting the applications' due dates. STS optimizes the costs and resource usage by starting with low-power configurations and progressively increasing the assigned resources to recover potential delays.…”

“…• We consider stochastic models for the jobs training times, while the previous work assumed deterministic estimates based on executing a fixed number of epochs (a less realistic assumption in real-life scenarios). • We develop an original heuristic method, STS, which incorporates GPU sharing and leverages the solution model proposed in [17] to tackle the inherent stochasticity of the optimal schedule. • We design a simulator to test the quality of solutions determined by STS, capable of dealing with training times stochasticity.…”