Holistic Slowdown Driven Scheduling and Resource Management for Malleable Jobs

Abstract

In job scheduling, the concept of malleability has been explored since many years ago. Research shows that malleability improves system performance, but its utilization in HPC never became widespread. The causes are the difficulty in developing malleable applications, and the lack of support and integration of the different layers of the HPC software stack. However, in the last years, malleability in job scheduling is becoming more critical because of the increasing complexity of hardware and workloads. In this context, using nodes in an exclusive mode is not always the most efficient solution as in traditional HPC jobs, where applications were highly tuned for static allocations, but offering zero flexibility to dynamic executions. This paper proposes a new holistic, dynamic job scheduling policy, Slowdown Driven (SD-Policy), which exploits the malleability of applications as the key technology to reduce the average slowdown and response time of jobs. SD-Policy is based on backfill and node sharing. It applies malleability to running jobs to make room for jobs that will run with a reduced set of resources, only when the estimated slowdown improves over the static approach. We implemented SD-Policy in SLURM and evaluated it in a real production environment, and with a simulator using workloads of up to 198K jobs. Results show better resource utilization with the reduction of makespan, response time, slowdown, and energy consumption, up to respectively 7%, 50%, 70%, and 6%, for the evaluated workloads. CCS CONCEPTS• Software and its engineering → Scheduling; • Computer systems organization → Multicore architectures.

Energy hardware and workload aware job scheduling towards interconnected HPC environments

“…Sarood et al [20] combine malleability and DVFS to create a scheduling policy that adapts the workload to a strict power budget in over-provisioned systems. Similarly, in a precedent research [21], [22], we used malleability and node sharing techniques to reduce response time, makespan, and energy consumption. The integration of the policies is a promising research direction.…”

“…Users are still in charge of specifying the number of requested nodes and processes. The number of threads or processes per node can be configured by using Slurm plugins or precedent work [21], [22] more intelligently. If a memory requirement is specified and some nodes cannot satisfy this requirement, a new number of nodes will be calculated.…”

nOS-V: Co-Executing HPC Applications Using System-Wide Task Scheduling

“…For HPC workloads, static co-location [7,17,52,54] has been proposed to reduce application interference. With static co-location, system resources are statically partitioned, and each application runs confined in a specific partition.…”

“…All approaches to improve node-level efficiency are translated into cluster-level improvements by fitting batch schedulers like SLURM [53] with job co-scheduling capabilities that consider how node-level resources are shared [17,23,34,52,54,55]. The coscheduling problem is related to node-level efficiency: better alternatives to in-node resource sharing can be combined with existing scheduling policies to deliver better cluster-level efficiency.…”

A Resourceful Coordination Approach for Multilevel Scheduling

“…In practice, production RJMS systems, such as Slurm [7] and Torque [8], support static resource allocation (SRA), where the number of the allocated resources is defined before the job execution and cannot subsequently be changed. Certain research efforts have extended production RJMS to support malleability [9,10].…”