Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling

Talby, David; Feitelson, Dror G.

doi:10.1109/ipps.1999.760525

Cited by 93 publications

(75 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of optimal packing is NP-complete, but several effective heuristic techniques have been developed. Examples of recent work in non-preemptive, preemptive and multiple resource schemes include [8,10,4,6]. Particular scheduling strategies depend on the availability of certain key system functionalities.…”

Section: Scheduling and Utilizationmentioning

confidence: 99%

ESP: A System Utilization Benchmark

Wong¹,

Oliker²,

Kramer³

et al. 2000

ACM/IEEE SC 2000 Conference (SC'00)

View full text Add to dashboard Cite

This article describes a new benchmark, called the Effective System Performance (ESP) test, which is designed to measure system-level performance, including such factors as job scheduling efficiency, handling of large jobs and shutdown-reboot times. In particular, this test can be used to study the effects of various scheduling policies and parameters. We present here some results that we have obtained so far on the Cray T3E and IBM SP systems, together with insights obtained from simulations. IntroductionThe overall performance value of a high performance computing system depends not only on its raw computational speed but also on system management effectiveness, including job scheduling efficiency, reboot and recovery times and the level of process management. Common performance metrics such as the LINPACK and NAS Parallel Benchmarks [3,1] are useful for measuring sustained computational performance for individual jobs, but give little or no insight into system-level efficiency issues.In this article, we describe a new benchmark, the Effective System Performance (ESP) benchmark, which measures system utilization and effectiveness [9]. Our primary motivation in developing this benchmark is to aid the evaluation of high performance systems. We plan to use it to monitor the impact of configuration changes and software upgrades in existing systems. But we also hope that this benchmark will provide a focal point for future research and development activities in the high performance computing community, possibly leading to significantly improved system-level efficiency in future production systems.The ESP test extends the idea of a throughput benchmark with additional features that mimic dayto-day supercomputer center operation. It yields an efficiency measurement based on the ratio of the actual elapsed time relative to the theoretical minimum time assuming perfect efficiency. This ratio is independent of the computational rate and is also relatively independent of the number of processors used, thus permitting comparisons between platforms.

show abstract

Section: Scheduling and Utilizationmentioning

confidence: 99%

ESP: A System Utilization Benchmark

Wong¹,

Oliker²,

Kramer³

et al. 2000

ACM/IEEE SC 2000 Conference (SC'00)

View full text Add to dashboard Cite

show abstract

“…Submitted jobs are judicially allocated by MUSCLE to suitable clusters for further scheduling and execution. The PACE (performance analysis and characterisation environment) toolkit [8,18] is incorporated into the architecture to provide execution time predictions for the submitted jobs.…”

Section: System and Workload Modelmentioning

confidence: 99%

“…Therefore, over-deadline can be used to measure the extent to which the QoS demands of a job set is satisfied, where over-deadline is defined as the sum of the excess time of each job's finish time over its deadline. The scheduling of parallel jobs has been studied extensively in single cluster environments [4,[6][7][8]. In such a scenario, submitted jobs typically have no QoS requirements and resource utilisation is a commonly used system-oriented metric [4].…”

Section: Introductionmentioning

confidence: 99%

Hybrid performance-based workload management for multiclusters and grids

Jarvis

Spooner

et al. 2004

IEE Proc., Softw.

View full text Add to dashboard Cite

The paper addresses the dynamic scheduling of parallel jobs with quality-of-service demands (soft-deadlines) in multiclusters and grids. Three performance metrics (over-deadline, makespan and idle-time) are combined with variable weights to evaluate the scheduling performance. These three metrics are used to measure the extent to which jobs comply with their QoS demands, the resource throughput and the resource utilisation. Therefore, clusters situated in different administrative organisations can utilise different weight combinations to represent their different performance requirements. Two levels of performance optimisation are applied in the multicluster. At the multicluster level, a scheduler (which we call MUSCLE) allocates parallel jobs with high packing potential to the same cluster; MUSCLE also takes the jobs' QoS requirements into account and employs a heuristic to allocate suitable workloads to each cluster to balance the overall system performance. At the local cluster level, an existing workload manager, called TITAN, utilises a genetic algorithm to further improve the scheduling performance of the jobs sent by MUSCLE. Extensive experimental studies are conducted to verify the effectiveness of the scheduling mechanism in MUSCLE. The results show that, compared with traditional distributed workload allocation policies, the comprehensive scheduling performance (in terms of over-deadline, makespan and idle-time) of parallel jobs is significantly improved across the multicluster.

show abstract

“…Not surprisingly this problem is NP-hard and many heuristic techniques have been developed to approximate a good solution. Examples of recent work for non-preemptive, preemptive and multiple resource schemes include [4,6,5,7].…”

Section: Utilization and Schedulingmentioning

confidence: 99%

System Utilization Benchmark on the Cray T3E and IBM SP

Wong

Oliker

Kramer

et al. 2000

Job Scheduling Strategies for Parallel Processing

View full text Add to dashboard Cite

Abstract.Obtaining maximum utilization of parallel systems continues to be an active area of research and development. This article outlines a new benchmark, called the Eflectiwe System Performance (ESP) test, designed to provide a utilization metric that is transferable between systems and illuminate the effects of various scheduling parameters. Results with discussion are presented for the Cray T3E and IBM SP systems together with insights obtained from simulation.

show abstract

Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling

Cited by 93 publications

References 10 publications

ESP: A System Utilization Benchmark

ESP: A System Utilization Benchmark

Hybrid performance-based workload management for multiclusters and grids

System Utilization Benchmark on the Cray T3E and IBM SP

Contact Info

Product

Resources

About