Improving HPC System Throughput and Response Time using Memory Disaggregation

Abstract

HPC clusters are cost-effective, well understood, and scalable, but the rigid boundaries between compute nodes may lead to poor utilization of compute and memory resources. HPC jobs may vary, by orders of magnitude, in memory consumption per core. Thus, even when the system is provisioned to accommodate normal and large capacity nodes, a mismatch between the system and the memory demands of the scheduled jobs can lead to inefficient usage of both memory and compute resources.Disaggregated memory has recently been proposed as a way to mitigate this problem by flexibly allocating memory capacity across cluster nodes. This paper presents a simulation approach for at-scale evaluation of job schedulers with disaggregated memories and it introduces a new disaggregated-aware job allocation policy for the Slurm resource manager. Our results show that using disaggregated memories, depending on the imbalance between the system and the submitted jobs, a similar throughput and job response time can be achieved on a system with up to 33% less total memory provisioning.

Memory Demands in Disaggregated HPC: How Accurate Do We Need to Be?

“…We use the BSC simulator extensions for disaggregated memory [9], [10], which adds support for disaggregated memory, modelling the slowdown due to remote memory accesses, and extends Slurm's allocation policy to exploit disaggregated memory. The performance model is a slowdownbased method [22], extended to support MPI processes [9].…”

“…We use the BSC simulator extensions for disaggregated memory [9], [10], which adds support for disaggregated memory, modelling the slowdown due to remote memory accesses, and extends Slurm's allocation policy to exploit disaggregated memory. The performance model is a slowdownbased method [22], extended to support MPI processes [9]. The allocation policy first selects nodes that have enough local memory to satisfy the job's requirement of memory per node to avoid unnecessary remote memory access.…”

“…Our analysis uses Barcelona Supercomputing Center's (BSC) scalable Slurm simulator for disaggregated memory systems (see Section II-C) [9], [10].…”

Job scheduling for disaggregated memory in high performance computing systems

“…It also details the hardware used to run our experiments, the set of single and multi node applications we profiled to create the contention model, and the concept of disaggregated memory employed in this work. Much of the content in this Chapter is presented in our papers [39,40].…”

“…We also detail the methodology applied to generate the workload employed in all our experiments. The methodology detailed in this Chapter is employed in our papers [40][41][42].…”