A HPC Co-scheduler with Reinforcement Learning

Abstract

Although High Performance Computing (HPC) users understand basic resource requirements such as the number of CPUs and memory limits, internal infrastructural utilization data is exclusively leveraged by cluster operators, who use it to configure batch schedulers. This task is challenging and increasingly complex due to ever larger cluster scales and heterogeneity of modern scientific workflows. As a result, HPC systems achieve low utilization with long job completion times (makespans). To tackle these challenges, we propose a co-scheduling algorithm based on an adaptive reinforcement learning algorithm, where application profiling is combined with cluster monitoring. The resulting cluster scheduler matches resource utilization to application performance in a fine-grained manner (i.e., operating system level). As opposed to nominal allocations, we apply decision trees to model applications' actual resource usage, which are used to estimate how much resource capacity from one allocation can be co-allocated to additional applications. Our algorithm learns from incorrect co-scheduling decisions and adapts from changing environment conditions, and evaluates when such changes cause resource contention that impacts quality of service metrics such as jobs slowdowns. We integrate our algorithm in an HPC resource manager that combines Slurm and Mesos for job scheduling and co-allocation, respectively. Our experimental evaluation performed in a dedicated cluster executing a mix of four real different scientific workflows demonstrates improvements on cluster utilization of up to 51% even in high load scenarios, with 55% average queue makespan reductions under low loads.

RIFLING: A reinforcement learning‐based GPU scheduler for deep learning research and development platforms

“…Moreover, RL are widely adopted in resource management and job scheduling. [19][20][21][22][23][24][25] They focus on the general scenarios like HPC clusters and datacenters where the workloads are diverse. The workloads are always characterized by the resource requirements, including memory, CPU, bandwidth, storage.…”

“…In addition, research 36 shows that an RL‐based approach can outperform other hand‐crafted methods in the network architecture search (NAS) field, while researches 15‐18 proved that RL techniques can be applied to solve additional problems related to communication optimization and network research. Moreover, RL are widely adopted in resource management and job scheduling 19‐25 . They focus on the general scenarios like HPC clusters and datacenters where the workloads are diverse.…”

“…Prior work 21,22 pays more attention to introducing deep reinforcement learning (Deep RL) into resource management of large-scale and complex systems. Moreover, recent studies [23][24][25] focus on HPC and cloud computing scenarios to schedule data-processing workloads with RL-based methods. In this article, we pursue a practical RL-based approach for DL task placement that embraces online scheduling without any manual intervention.…”