Dynamic fractional resource scheduling for HPC workloads

Stillwell, Mark; Vivien, Frédéric; Casanova, Henri

doi:10.1109/ipdps.2010.5470356

Cited by 25 publications

(15 citation statements)

References 34 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assume that a computer can host one or multiple jobs. This model is similar to the work of Stillwell et al [13]. This kind of jobs is common: a compute node can run multiple MapReduce tasks; an online game hoster may consolidate several game servers on the same machine; etc.…”

Section: Workload and Resource Modelmentioning

confidence: 94%

Scheduling Jobs in the Cloud Using On-Demand and Reserved Instances

Shen

Deng

Iosup

et al. 2013

Euro-Par 2013 Parallel Processing

View full text Add to dashboard Cite

Abstract. Deploying applications in leased cloud infrastructure is increasingly considered by a variety of business and service integrators. However, the challenge of selecting the leasing strategy -larger or faster instances? on-demand or reserved instances? etc.-and to configure the leasing strategy with appropriate scheduling policies is still daunting for the (potential) cloud user. In this work, we investigate leasing strategies and their policies from a broker's perspective. We propose, CoH, a family of Cloud-based, online Hybrid scheduling policies that minimizes rental cost by making use of both on-demand and reserved instances. We formulate the resource provisioning and job allocation policies as Integer Programming problems. As the policies need to be executed online, we limit the time to explore the optimal solution of the integer program, and compare the obtained solution with various heuristics-based policies; then automatically pick the best one. We show, via simulation and using multiple real-world traces, that the hybrid leasing policy can obtain significantly lower cost than typical heuristics-based policies.

show abstract

Section: Workload and Resource Modelmentioning

confidence: 94%

Scheduling Jobs in the Cloud Using On-Demand and Reserved Instances

Shen

Deng

Iosup

et al. 2013

Euro-Par 2013 Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Particularly, the work by Stillwell [14] addressed a set of scheduling issues. Stillwell's work proposed job placement methods based on current CPU utilization and memory occupancy using Multi-Capacity Bin Packing (MCB) algorithms.…”

Section: Related Workmentioning

confidence: 99%

Efficiency Assessment of Parallel Workloads on Virtualized Resources

Delgado

Sadjadi

Fong

et al. 2011

2011 Fourth IEEE International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

Abstract-In cloud computing, virtual containers on physical resources are provisioned to requesting users. Resource providers may pack as many containers as possible onto each of their physical machines, or may pack specific types and quantities of virtual containers based on user or system QoS objectives. Such elastic provisioning schemes for resource sharing may present major challenges to scientific parallel applications that require task synchronization during execution. Such elastic schemes may also inadvertently lower utilization of computing resources. In this paper, we describe the elasticity constraint effect and ripple effect that cause a negative impact to application response time and system utilization. We quantify the impact using real workload traces through simulation. Then, we demonstrate that some resource scheduling techniques can be effective in mitigating the impacts. We find that a tradeoff is needed among the elasticity of virtual containers, the complexity of scheduling algorithms, and the response time of applications.General Terms: scheduling, virtualization, parallel application I. INTRODUCTION The current trend in resource provisioning for utility and cloud computing is to provide compute resources as virtual resource containers using virtualization technologies. The most common realization is the use of virtual machine (VM) technology, in which a virtual container manager (VCM) abstracts and controls the physical resources allotted to one or more VMs. The VM abstraction simplifies the deployment of application environments across a wide range of physical systems. The VCM provides fine-grained control of the amount of shared resources to apportion to each VM when they are competing for them. This provides resource providers flexible resource allocation control to satisfy quality of service of requests and to build cost and profit models [1]. This flexibility also allows application execution to be tuned in different ways. For example, execution of a job proceeds faster than expected when its VM borrows idle CPU cycles from collocated VMs. Moreover, parallel applications need not limit the degree of parallelism to the number of physical machines.The simplicity of application deployment and the flexibility of virtual resource acquisition enabled by the Cloud is creating interest among users of CPU-intensive parallel applications [2]. However, this flexibility can present a unique challenge when scheduling and executing them.

show abstract

“…As a case in point, gang scheduling [12] has demonstrated improved scheduling performance, although it can lead to resource fragmentation if the total number of tasks is not a multiple of the number of available processors. Similarly, in [13] and [14], the authors simulate a scheduling algorithm that leverages CPU sharing, and average job response times improved by a factor of 8 or more, depending on specific optimizations, compared to only running one job at a time per machine using the popular EASY job scheduling algorithm [15].…”

Section: I14 the Role Of Virtualizationmentioning

confidence: 99%

“…For example, Stillwell [13][14] evaluated the use of dynamic fractional scheduling via simulation using real-world traces, while our work provides empirical results and a mathematical model to align such simulations with real-world data.…”

Section: Ii22 Virtualization-related Studiesmentioning

confidence: 99%