The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance

Chiang, Su-Hui; Arpaci-Dusseau, Andrea C.; Vernon, Mary K.

doi:10.1007/3-540-36180-4_7

Cited by 88 publications

(67 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Especially, users grossly overestimate their jobs' runtime. For instance, 50 to 60% of jobs use less than 20% of their requested time in four different traces [26], jobs on a Cray T3E use on average only 29% of their requested time [27], and 35% of jobs run less than 10% of their requested time [28]. Similar patterns are also seen in other recent workload analyses [1].…”

Section: Adaptive Allocationsupporting

confidence: 78%

Job Mapping and Scheduling on Free-Space Optical Networks

Fujiwara

Koibuchi

2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYA number of parallel applications run on a highperformance computing (HPC) system simultaneously. Job mapping and scheduling become crucial to improve system utilization, because fragmentation prevents an incoming job from being assigned even if there are enough compute nodes unused. Wireless supercomputers and datacenters with free-space optical (FSO) terminals have been proposed to replace the conventional wired interconnection so that a diverse application workload can be better supported by changing their network topologies. In this study we firstly present an efficient job mapping by swapping the endpoints of FSO links in a wireless HPC system. Our evaluation shows that an FSOequipped wireless HPC system can achieve shorter average queuing length and queuing time for all the dispatched user jobs. Secondly, we consider the use of a more complicated and enhanced scheduling algorithm, which can further improve the system utilization over different host networks, as well as the average response time for all the dispatched user jobs. Finally, we present the performance advantages of the proposed wireless HPC system under more practical assumptions such as different cabinet capacities and diverse subtopology packings.

show abstract

Section: Adaptive Allocationsupporting

confidence: 78%

Job Mapping and Scheduling on Free-Space Optical Networks

Fujiwara

Koibuchi

2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…More sophisticated approaches propose dynamic backfilling priorities based on the current wait time of the job and the job size (LXWF-Backfilling [4]). -The order in which the jobs are moved to the head of the wait queue, i.e.…”

Section: Related Workmentioning

confidence: 99%

“…The goal was to evaluate the performance of these policies with specific workloads in HPC centers. A special effort has been devoted to evaluating backfillingbased ( [4] [22]) policies because they have demonstrated an ability to reach the best performance results (i.e: [12] or [21]). Almost all of these studies have been done using simulation tools.…”

Section: Introductionmentioning

confidence: 99%

The Resource Usage Aware Backfilling

Guim

Rodero

Corbalan

2009

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. Job scheduling policies for HPC centers have been extensively studied in the last few years, especially backfilling based policies. Almost all of these studies have been done using simulation tools. All the existent simulators use the runtime (either estimated or real) provided in the workload as a basis of their simulations. In our previous work we analyzed the impact on system performance of considering the resource sharing (memory bandwidth) of running jobs including a new resource model in the Alvio simulator. Based on this studies we proposed the LessConsume and LessConsume Threshold resource selection policies. Both are oriented to reduce the saturation of the shared resources thus increasing the performance of the system. The results showed how both resource allocation policies shown how the performance of the system can be improved by considering where the jobs are finally allocated. Using the LessConsume Threshold Resource Selection Policy, we propose a new backfilling strategy : the Resource Usage Aware Backfilling job scheduling policy. This is a backfilling based scheduling policy where the algorithms which decide which job has to be executed and how jobs have to be backfilled are based on a different Threshold configurations. This backfilling variant that considers how the shared resources are used by the scheduled jobs. Rather than backfilling the first job that can moved to the run queue based on the job arrival time or job size, it looks ahead to the next queued jobs, and tries to allocate jobs that would experience lower penalized runtime caused by the resource sharing saturation. In the paper we demostrate how the exchange of scheduling information between the local resource manager and the scheduler can improve substantially the performance of the system when the resource sharing is considered. We show how it can achieve a close response time performance that the shorest job first Backfilling with First Fit (oriented to improve the start time for the allocated jobs) providing a qualitative improvement in the number of killed jobs and in the percentage of penalized runtime.

show abstract

“…The net effect of shortening the queue gives advantage to jobs that have not yet been submitted since they find a shorter queue. Studies by Chiang et al [6] show that in aggressive backfilling, rarely does a backfilled job delay jobs deep into the queue. This implies that while the disadvantage affects a few jobs at the head of the queue, the advantages go beyond the queue.…”

Section: Relative Performance Of Job Groupsmentioning

confidence: 99%

Group-Wise Performance Evaluation of Processor Co-allocation in Multi-cluster Systems

Ngubiri

Vliet

2008

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract. Performance evaluation in multi-cluster processor co-allocation -like in many other parallel job scheduling problems-is mostly done by computing the average metric value for the entire job stream. This does not give a comprehensive understanding of the relative performance of the different jobs grouped by their characteristics. It is however the characteristics that affect how easy/hard jobs are to schedule. We, therefore, do not get to understand scheduler performance at job type level. In this paper, we study the performance of multi-cluster processor co-allocation for different job groups grouped by their size, components and widest component. We study their relative performance, sensitivity to parameters and how their performance is affected by the heuristics used to break them up into components. We show that the widest component us characteristic that most affects job schedulability. We also show that to get better performance, jobs should be broken up in such a way that the width of the widest component is minimized.

show abstract

The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance

Cited by 88 publications

References 11 publications

Job Mapping and Scheduling on Free-Space Optical Networks

Job Mapping and Scheduling on Free-Space Optical Networks

The Resource Usage Aware Backfilling

Group-Wise Performance Evaluation of Processor Co-allocation in Multi-cluster Systems

Contact Info

Product

Resources

About