Parallel Job Scheduling — A Status Report

Feitelson, Dror G.; Rudolph, Larry; Schwiegelshohn, Uwe

doi:10.1007/11407522_1

Cited by 191 publications

(139 citation statements)

References 52 publications

(53 reference statements)

Supporting

Mentioning

137

Contrasting

Unclassified

Order By: Relevance

“…Job slowdown (S) and job wait time (W ) are used as common proxies [3] for user objectives. We also measure the total run time of all the jobs (R J ) and the total run time of all rented VM instances (R V ).…”

Section: Performance Metricsmentioning

confidence: 99%

“…Especially when servicing the demanding workloads typical of scientific computing [1,2], these data centers need efficient algorithms for scheduling their users' workloads on the data center resources. Many existing scheduling algorithms have already addressed specific workload properties [3,4] and types of applications [5][6][7][8], but data centers still rely on (expensive) human system administrators to select a scheduling algorithm and configure it appropriately. Moreover, the selection process is made significantly more difficult by changing workloads due to technology transitions (e.g., the use of virtualization and new networking architectures), and by new customers starting to use data centers as Infrastructure-as-a-Service clouds.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

Deng

Verboon

Ren

et al. 2014

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. The popularity of data centers in scientific computing has led to new architectures, new workload structures, and growing customerbases. As a consequence, the selection of efficient scheduling algorithms for the data center is an increasingly costlier and more difficult challenge. To address this challenge, and contrasting previous work on scheduling for scientific workloads, we focus in this work on portfolio schedulinghere, the dynamic selection and use of a scheduling policy, depending on the current system and workload conditions, from a portfolio of multiple policies. We design a periodic portfolio scheduler for the workload of the entire data center, and equip it with a portfolio of resource provisioning and allocation policies. Through simulation based on real and synthetic workload traces, we show evidence that portfolio scheduling can automatically select the scheduling policy to match both user and data center objectives, and that portfolio scheduling can perform well in the data center, relative to its constituent policies.

show abstract

Section: Performance Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

Deng

Verboon

Ren

et al. 2014

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Most research on parallel job scheduling had been focused on single Shared Memory computers, Distributed Memory Multiprocessors and clusters [10] [23]. Recently, research work has been extended to computational and data grids [13][27] as well as multi-cluster systems [4] [15].…”

Section: Introductionmentioning

confidence: 99%

Group-Wise Performance Evaluation of Processor Co-allocation in Multi-cluster Systems

Ngubiri

Vliet

2008

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. Performance evaluation in multi-cluster processor co-allocation -like in many other parallel job scheduling problems-is mostly done by computing the average metric value for the entire job stream. This does not give a comprehensive understanding of the relative performance of the different jobs grouped by their characteristics. It is however the characteristics that affect how easy/hard jobs are to schedule. We, therefore, do not get to understand scheduler performance at job type level. In this paper, we study the performance of multi-cluster processor co-allocation for different job groups grouped by their size, components and widest component. We study their relative performance, sensitivity to parameters and how their performance is affected by the heuristics used to break them up into components. We show that the widest component us characteristic that most affects job schedulability. We also show that to get better performance, jobs should be broken up in such a way that the width of the widest component is minimized.

show abstract

“…Traditional FIFO and backfilling parallel schedulers (surveyed in [8] and [9]) assume that data is already pre-staged and available to the application executables on the compute nodes, while workflow schedulers consider only the precedence relationship between the applications and the data and do not consider optimisation, e.g. [13].…”

Section: Introductionmentioning

confidence: 99%

Evolving Toward the Perfect Schedule: Co-scheduling Job Assignments and Data Replication in Wide-Area Systems Using a Genetic Algorithm

Phan

Ranganathan²,

Sion

2005

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Traditional job schedulers for grid or cluster systems are responsible for assigning incoming jobs to compute nodes in such a way that some evaluative condition is met. Such systems generally take into consideration the availability of compute cycles, queue lengths, and expected job execution times, but they typically do not account directly for data staging and thus miss significant associated opportunities for optimisation. Intuitively, a tighter integration of job scheduling and automated data replication can yield significant advantages due to the potential for optimised, faster access to data and decreased overall execution time. In this paper we consider data placement as a first-class citizen in scheduling and use an optimisation heuristic for generating schedules. We make the following two contributions. First, we identify the necessity for co-scheduling job dispatching and data replication assignments and posit that simultaneously scheduling both is critical for achieving good makespans. Second, we show that deploying a genetic search algorithm to solve the optimal allocation problem has the potential to achieve significant speed-up results versus traditional allocation mechanisms. Through simulation, we show that our algorithm provides on average an approximately 20-45% faster makespan than greedy schedulers.

show abstract

Parallel Job Scheduling — A Status Report

Cited by 191 publications

References 52 publications

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

Group-Wise Performance Evaluation of Processor Co-allocation in Multi-cluster Systems

Evolving Toward the Perfect Schedule: Co-scheduling Job Assignments and Data Replication in Wide-Area Systems Using a Genetic Algorithm

Contact Info

Product

Resources

About