ANDREAS: Artificial intelligence traiNing scheDuler foR accElerAted resource clusterS

Filippini, Federica; Ardagna, Danilo; Lattuada, Marco; Amaldi, E.; Riedl, Maciek; Materka, Katarzyna; Skrzypek, Pawel; Ciavotta, Michele; Magugliani, Fabrizio; Cicala, Marco

doi:10.1109/ficloud49777.2021.00063

Cited by 6 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assume, as in [10], [31], that the (instantaneous) power consumption when operating at speed s k takes the form:…”

Section: B Power-consumption Modelmentioning

confidence: 99%

“…Specifically, at most one fraction in F can be assigned to each job (see (8)), the sum of the fractions assigned to all jobs in a specific GPU g must not exceed 1 (see ( 9)), and the fraction f = 1 must be assigned to all jobs that are executed on more than one GPU (identified by the sum at left-hand side of Constraints (10)).…”

Section: Resource Selection-job Scheduling Problemmentioning

confidence: 99%

“…This work aims to optimize the resource selection and scheduling of DL training jobs from the perspective of a CSP running a data center, efficiently selecting resources for the execution of each job to minimize the energy consumption costs while meeting the applications' due dates. To the best of our knowledge, methods available in the literature tackle this problem by proposing (i) simple job scheduling mechanisms, such as Earliest-Deadline-First (EDF) or First-in-First-Out (FIFO) [3]- [6], possibly coupled with effective resource selection algorithms, or (ii) more elaborate heuristics [7]- [9], sometimes even coupling the resource selection and scheduling problem as in our previous works [10], [11]. Albeit achieving good-quality solutions, these approaches only consider the worst-case execution times of DL applications in searching for an optimal schedule.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Stochastic Approach for Scheduling AI Training Jobs in GPU-Based Systems

Filippini,

Anselmi,

Ardagna

et al. 2024

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To model the problem, we first develop a Mixed-Integer Non-Linear Programming formulation. Unfortunately, the computation of an optimal solution is prohibitively expensive, and to overcome this difficulty, we design a heuristic STochastic Scheduler (STS). Exploiting the probability distribution of early termination, STS determines how to adapt the resource assignment during the execution of the jobs to minimize the expected energy cost while meeting the job due dates. The results of an extensive experimental evaluation show that STS guarantees significantly better results than other methods in the literature, effectively avoiding due date violations and yielding a percentage total cost reduction between 32% and 80% on average. We also prove the applicability of our method in real-world scenarios, as obtaining optimal schedules for systems of up to 100 nodes and 400 concurrent jobs requires less than 5 seconds. Finally, we evaluated the effectiveness of GPU sharing, i.e., running multiple jobs in a single GPU. The obtained results demonstrate that depending on the workload and GPU memory, this further reduces the energy cost by 17-29% on average.

show abstract

“…We assume, as in [10], [31], that the (instantaneous) power consumption when operating at speed s k takes the form:…”

Section: B Power-consumption Modelmentioning

confidence: 99%

Section: Resource Selection-job Scheduling Problemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Stochastic Approach for Scheduling AI Training Jobs in GPU-Based Systems

Filippini,

Anselmi,

Ardagna

et al. 2024

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

show abstract

“…Existing GPU datacenters have considerable power waste as not all the GPUs are actively used all the time, while the datacenter managers prefer to keep all the devices on. To reduce the energy cost, ANDREAS [39] considers a scenario where the execution of each job can be postponed within a certain period. Then it judiciously schedules jobs at appropriate moments to keep all the GPUs busy in the datacenter.…”

Section: Efficiencymentioning

confidence: 99%

Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision

Gao¹,

Hu²,

Ye³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning (DL) shows its prosperity in a wide variety of fields. The development of a DL model is a time-consuming and resource-intensive procedure. Hence, dedicated GPU accelerators have been collectively constructed into a GPU datacenter. An efficient scheduler design for such GPU datacenter is crucially important to reduce the operational cost and improve resource utilization. However, traditional approaches designed for big data or high performance computing workloads can not support DL workloads to fully utilize the GPU resources. Recently, substantial schedulers are proposed to tailor for DL workloads in GPU datacenters. This paper surveys existing research efforts for both training and inference workloads. We primarily present how existing schedulers facilitate the respective workloads from the scheduling objectives and resource consumption features. Finally, we prospect several promising future research directions. More detailed summary with the surveyed paper and code links can be found at our project website: https://github.com/S-Lab-System-Group/Awesome-DL-Scheduling-Papers. CCS Concepts: • General and reference → Surveys and overviews; • Computing methodologies → Machine learning; • Computer systems organization → Cloud computing.

show abstract