A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

Ilager, Shashikant; Muralidhar, Rajeev; Rammohanrao, Kotagiri; Buyya, Rajkumar

doi:10.1109/ccgrid49817.2020.00-35

Cited by 7 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, our work represents one of the first attempts to tackle the joint problem of online DL job scheduling and resource allocation on multiple virtualized GPUs. As mentioned in Section 1, most of the available literature proposals focus on either the scheduling or the resource selection aspect, leaving the decisions on the number and type of GPUs to the users (e.g., [6], [7]) or delegating the job scheduling to simple mechanisms as FIFO or EDF (e.g., [8], [9]), respectively. Therefore, we will briefly review part of the existing literature in the two scenarios.…”

Section: Related Workmentioning

confidence: 99%

“…However, in this work jobs are subject to hard deadlines and Virtual Machines are rented from the public cloud to scale the available resources and avoid violations. A data-driven Dynamic Voltage Frequency Scaling method is exploited in [8] to guide a deadline-aware scheduling algorithm based on an EDF approach, aiming to maximize the energy e ciency of a cluster. Inter-user fairness is the main goal pursued by Gandiva fair [14].…”

Section: Jobs Schedulingmentioning

confidence: 99%

“…To the best of our knowledge, most of the existing literature focuses on either the scheduling or the resource selection aspect, relying on users' requests in terms of resources (i.e., specifying the type and number of GPUs) to assign to the submitted workloads (see, e.g., [6], [7]) or delegating the job scheduling to simple mechanisms (e.g., First In First Out; see, for instance, [8], [9]), respectively. In this work, we tackle the online resource selection and job scheduling problems jointly and from a higher level perspective, where users are only asked to specify: (i) a due date for each DL training job, (ii) a priority associated with the fulfillment of the due date.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Path Relinking Method for the Joint Online Scheduling and Capacity Allocation of DL Training Workloads in GPU as a Service Systems

Filippini

Lattuada

Ciavotta

et al. 2023

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

The Deep Learning (DL) paradigm gained remarkable popularity in recent years. DL models are used to tackle increasingly complex problems, making the training process require considerable computational power. The parallel computing capabilities o ered by modern GPUs partially fulfill this need, but the high costs related to GPU as a Service solutions in the cloud call for e cient capacity planning and job scheduling algorithms to reduce operational costs via resource sharing. In this work, we jointly address the online capacity planning and job scheduling problems from the perspective of cloud end-users. We present a Mixed Integer Linear Programming (MILP) formulation, and a path relinking-based method aiming at optimizing operational costs by (i) rightsizing Virtual Machine (VM) capacity at each node, (ii) partitioning the set of GPUs among multiple concurrent jobs on the same VM, and (iii) determining a due-date-aware job schedule. An extensive experimental campaign attests the e ectiveness of the proposed approach in practical scenarios: costs savings up to 97% are attained compared with first-principle methods based on, e.g., Earliest Deadline First, cost reductions up to 20% are obtained with respect to a previously proposed Hierarchical Method and up to 95% against a dynamic programming-based method from the literature. Scalability analyses show that systems with up to 100 nodes and 450 concurrent jobs can be managed in less than 7 seconds. The validation in a prototype cloud environment shows a deviation below 5% between real and predicted costs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Jobs Schedulingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Path Relinking Method for the Joint Online Scheduling and Capacity Allocation of DL Training Workloads in GPU as a Service Systems

Filippini

Lattuada

Ciavotta

et al. 2023

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

show abstract

“…A memory‐intensive application would require higher memory frequency compared to CPU frequencies, and such heterogeneity needs to be accounted for in the solution. In our recent work, 19 we proposed data‐driven ML techniques for an energy‐efficient frequency scaling of GPUs for commonly used benchmarking applications. We created prediction models that learn different applications' behaviour on different frequency settings, which helps the scheduler to configure energy‐efficient frequencies while meeting applications' QoS requirements.…”

Section: Research Issues and Envisioned Approaches As Future Directionsmentioning

confidence: 99%

Energy‐efficiency and sustainability in new generation cloud computing: A vision and directions for integrated management of data centre resources and workloads

Buyya,

Ilager,

Arroba

2023

Softw Pract Exp

Self Cite

View full text Add to dashboard Cite

Cloud computing has become a critical infrastructure for modern society, like electric power grids and roads. As the backbone of the modern economy, it offers subscription‐based computing services anytime, anywhere, on a pay‐as‐you‐go basis. Its use is growing exponentially with the continued development of new classes of applications driven by a huge number of emerging networked devices. However, the success of Cloud computing has created a new global energy challenge, as it comes at the cost of vast energy usage. Currently, data centres hosting Cloud services world‐wide consume more energy than most countries. Globally, by 2025, they are projected to consume 20% of global electricity and emit up to 5.5% of the world's carbon emissions. In addition, a significant part of the energy consumed is transformed into heat which leads to operational problems, including a reduction in system reliability and the life expectancy of devices, and escalation in cooling requirements. Therefore, for the future generations of Cloud computing to address the environmental and operational consequences of such significant energy usage, they must become energy‐efficient and environmentally sustainable while continuing to deliver high‐quality services. In this article, we propose a vision for learning‐centric approach for the integrated management of new generation Cloud computing environments to reduce their energy consumption and carbon footprint while delivering service quality guarantees. In this article, we identify the dimensions and key issues of integrated resource management and our envisioned approaches to address them. We present a conceptual architecture for energy‐efficient new generation Clouds and early results on the integrated management of resources and workloads that evidence its potential benefits towards energy efficiency and sustainability.

show abstract

“…Another hardware-based technique called dynamic voltage and fre-quency scaling (DVFS) [16] adjusts the GPU's operating voltage and frequency in accordance with workload demands. Energy usage can be optimized by dynamically adjusting these parameters to the ideal efficiency level [5]. In periods of reduced processing activity, DVFS permits the GPU to operate at lower voltages and frequencies, thus reducing power consumption while maintaining performance.…”

Section: Introductionmentioning

confidence: 99%

Efficient GPU Power Management through Advanced Framework Utilizing Optimization Algorithms

Rehman,

Chishti,

Yamin

2024

CSJM

View full text Add to dashboard Cite

The rapid rise in power usage by GPUs due to advances in machine and deep learning has led to an increase in power consumption of GPUs in Deep Learning workloads. To address this issue, a novel research project focuses on integrating Particle Swarm Optimization into a model training optimization framework to effectively reduce GPU power consumption during machine learning and deep learning training workloads. By utilizing the Particle Swarm Optimization (PSO)\protect\hyperlink{b1}{{[}1{]}} algorithm within the proposed framework, we show the effectiveness of PSO in creating a more efficient power management strategy while also maintaining the performance. Upon evaluation of the proposed framework, it shows a reduction of 15.8\% to 75.8\% in power consumption across multiple workloads, with little to no performance loss.

show abstract

A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)

Cited by 7 publications

References 23 publications

A Path Relinking Method for the Joint Online Scheduling and Capacity Allocation of DL Training Workloads in GPU as a Service Systems

A Path Relinking Method for the Joint Online Scheduling and Capacity Allocation of DL Training Workloads in GPU as a Service Systems

Energy‐efficiency and sustainability in new generation cloud computing: A vision and directions for integrated management of data centre resources and workloads

Efficient GPU Power Management through Advanced Framework Utilizing Optimization Algorithms

Contact Info

Product

Resources

About