RackMem

Abstract

High-performance computing (HPC) clusters suffer from an overall low memory utilization that is caused by the node-centric memory allocation combined with the variable memory requirements of HPC workloads. The recent provisioning of nodes with terabytes of memory to accommodate workloads with extreme peak memory requirements further exacerbates the problem. Memory disaggregation is viewed as a promising remedy to increase overall resource utilization and enable cost-effective up-scaling and efficient operation of HPC clusters, however, the overhead of demand paging in virtual memory management has so far hindered performant implementations. To overcome these limitations, this work presents RackMem, an efficient implementation of disaggregated memory for rack scale computing. RackMem addresses the shortcomings of Linux's demand paging algorithm and automatically adapts to the memory access patterns of individual processes to minimize the inherent overhead of remote memory accesses. Evaluated on a cluster with an Infiniband interconnect, RackMem outperforms the state-of-the-art RDMA implementation and Linux's virtual memory paging by a significant margin. RackMem's custom demand paging implementation achieves a tail latency that is two orders of magnitude better than that of the Linux kernel. Compared to the state-of-the-art remote paging solution, RackMem achieves a 28% higher throughput and a 44% lower tail latency for a wide variety of real-world workloads. CCS CONCEPTS • Computer systems organization → Cloud computing; • Software and its engineering → Virtual memory; Distributed memory; Cloud computing.

Job scheduling for disaggregated memory in high performance computing systems

“…The default node allocation of Slurm is the exclusive also known as server-based or node-based allocation mode. In this allocation mode a job is placed only if a node can satisfy the amount of requested core and memory resources [55], so even if not all resources within the node are utilized by a specific job, no other job is allowed to share the resource. As a matter of fact, many HPC systems disallow the colocation of different workloads on the same compute node to minimize the negative impact caused by inter-workload interference [56,10].…”

“…Modern HPC systems are therefore typically built from thousands of ccNUMA nodes communicating via a fast interconnect such as InfiniBand or OmniPath. However, they often suffer from memory underutilization [55].…”

“…According to [2,10,55], the fundamental reason is the current node-based memory allocation. Computing and memory resources are tightly coupled in each node, resources are requested in units of nodes, and job memory allocation is based on peak usage.…”

“…However, memory utilization varies widely from one job to another over time across the nodes, which results in underutilization when there are jobs with small memory footprint [10,55]. Additionally, many jobs on HPC systems also overestimate their memory demands, thus underutilizing memory slots dedicated to the specific jobs [2].…”

“…Jo et al [55] present better performance than [14] by introducing a novel RDMAbacked caching layer for memory disaggregation. They develop a custom paging module that automatically adapts to the memory access patterns of individual processes to minimize the inherent overhead of remote memory accesses.…”