TMO: transparent memory offloading in datacenters

Abstract

The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta's transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device's performance characteristics and the application's sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

MEMTIS: Efficient Memory Tiering with Dynamic Page Classification and Page Size Determination

“…TMTS primarily focuses on replacing a portion of its DRAM usage with the slower memory tier to reduce memory cost while minimizing performance impact (<5%), rather than expanding the system memory capacity. TMTS sets its target secondary tier residency ratio (STRR) to 25%, which aligns well with the cold memory ratio observed in WSCs [22,39,81]. The demotion and promotion policies of TMTS are designed to maintain secondary tier access ratio (STAR) of applications to remain within the target range (<0.5%), especially in scenarios where the hot working set of applications can fit within the fast tier.…”

COSMOS: Coordinated Management of Cores, Memory, and Compressed Memory Swap for QoS-Aware and Efficient Workload Consolidation for Memory-Intensive Applications

“…The memory demands in cloud computing systems and datacenters are explosively growing because of the rise of emerging memory-intensive applications such as machine learning and big data applications [38], [49]. In addition, DRAM scaling has slowed down [24], [28] and there have been large fluctuations in DRAM prices [46]. As a result, DRAM has become one of the most critical and expensive components in cloud computing systems and datacenters [46].…”

“…In addition, DRAM scaling has slowed down [24], [28] and there have been large fluctuations in DRAM prices [46]. As a result, DRAM has become one of the most critical and expensive components in cloud computing systems and datacenters [46].…”

“…The compressed memory swap (CMS) [40] is a promising technique to host memory-intensive applications without increasing the memory capacity of the underlying server system [23], [46]. With the CMS, pages selected as victim pages by the memory reclaim algorithm in the OS are compressed and evicted to the CMS instead of the disk swap.…”

“…The CMS incurs overheads for compressing and decompressing pages when pages are transferred between the memory and CMS. However, since the page compression and decompression operations are performed using CPU cores and memory, the CMS is significantly faster than the disk swap, which incurs expensive disk I/O operations [46]. The CMS is supported by widely-used OSes such as Windows [3], Linux [6], and macOS [5] and employed in commercial cloud computing systems and datacenters [23], [46].…”