Clio: a hardware-software co-designed disaggregated memory system

Abstract

Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems.Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system.We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 𝜇𝑠 at median and 3.2 𝜇𝑠 at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than softwarebased SmartNIC solutions.

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

“…To alleviate the interconnect (PCIe) overhead, a line of research integrates the entire NIC or accelerator to/near the CPU package [3,32,33,55,61,80,100,119,139]. Enjoying the benefit of fast NIC-core interaction, this approach has (1) high cost of designing and manufacturing them and (2) low flexibility of usage and maintaining.…”

DINOMO: An Elastic, Scalable, High-Performance Key-Value Store for Disaggregated Persistent Memory (Extended Version)

“…In active DPM, the compute capacity at DPM is used for local processing, including network, applicationlevel, and data store processing [41,56,70]. Prior work has proposed data stores for active DPM that leverage this limited computational power [29,53,56,76,90]. In contrast, passive DPM has no computational abilities at the DPM pool.…”

FUSEE: A Fully Memory-Disaggregated Key-Value Store (Extended Version)

“…Traditional in-memory key-value (KV) stores on monolithic servers have recently been ported to the disaggregated memory (DM) architecture for better resource efficiency [60,73]. Compared with monolithic servers, DM decouples the compute and memory resources into independent network-attached compute and memory pools [3,23,25,39,48,55,56,65]. KV stores on DM can thus enjoy efficient resource pooling and have higher resource efficiency.…”