The Case for RackOut

Abstract

To provide low latency and high throughput guarantees, most large key-value stores keep the data in the memory of many servers. Despite the natural parallelism across lookups, the load imbalance, introduced by heavy skew in the popularity distribution of keys, limits performance. To avoid violating tail latency service-level objectives, systems tend to keep server utilization low and organize the data in micro-shards, which provides units of migration and replication for the purpose of load balancing. These techniques reduce the skew, but incur additional monitoring, data replication and consistency maintenance overheads.In this work, we introduce RackOut, a memory pooling technique that leverages the one-sided remote read primitive of emerging rack-scale systems to mitigate load imbalance while respecting service-level objectives. In RackOut, the data is aggregated at rack-scale granularity, with all of the participating servers in the rack jointly servicing all of the rack's micro-shards. We develop a queuing model to evaluate the impact of RackOut at the datacenter scale. In addition, we implement a RackOut proof-of-concept key-value store, evaluate it on two experimental platforms based on RDMA and Scale-Out NUMA, and use these results to validate the model. Our results show that RackOut can increase throughput up to 6× for RDMA and 8.6× for Scale-Out NUMA compared to a scale-out deployment, while respecting tight tail latency service-level objectives.

Scale-out ccNUMA

“…To scale beyond a rack-scale or small cluster sized deployment, we believe our ideas can be applied by simply partitioning bigger deployments into smaller Scale-Out ccNUMA clusters, each of which can independently apply symmetric caching. For example, a KVS that spans 100 nodes can be split into five 20-machine groups (similar to [38]), where each group employs symmetric caching for its portion of the KVS. Resilience.…”

“…The exponent α is a function of the dataset and access pattern, and has been shown to lie close to unity. The most common value for α in recent literature is 0.99 [14,20,22,32,38], with 0.90 and 1.01 also frequently used and cited in KVS research [4,16]. An important implication of popularity skew is the resulting load imbalance across the set of servers maintaining the dataset.…”

“…NUMA abstraction: Pioneered in FaRM [14] and leveraged in RackOut [38], this approach offers a NUMA-like shared memory abstraction across the servers storing the dataset via remote access primitives over a low-latency RDMA-enabled network, as shown in Figure 2c. More specifically, the one-sided RDMA reads allow any server to directly access the memory of any other server in the deployment.…”

“…Although FaSST is designed for transaction processing, it has two key attributes that make it a good baseline for a system to tackle skew: it offers a NUMA abstraction like FaRM [14] and RackOut [38], and it leverages several design techniques to achieve high performance using RDMA [24]. We modify FaSST to implement efficient single-key Get and Put operations by stripping off all of the transaction processing overheads.…”

Sharing non‐cache‐coherent memory with bounded incoherence

“…GAM 44 provides a directory‐based cache coherence protocol over RDMA. Systems such as FaRM and RackOut 45‐47 treat a cluster as a non‐CC NUMA machine with RDMA‐accessible remote memory. They use similar techniques (e.g., epoch‐based memory reclamation 6 ), but don't support cached‐access to remote memory.…”