Architecting to achieve a billion requests per second throughput on a single key-value store server platform

Li, Sheng; Lim, Hyeontaek; Lee, Victor W.; Ahn, Jung Ho; Kalia, Anuj; Kaminsky, Michael; Andersen, David G.; Seongil, O; Lee, Sukhan; Dubey, Pradeep

doi:10.1145/2872887.2750416

Cited by 16 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this reason, we naturally assume that to execute a transaction, a client can communicate with the servers (but not with other clients) and a server communicates with a client only to respond to a client's read or write request. We find evidence of the relevance of this assumption in large-scale production systems, such as Facebook's data platform [45], and in emerging systems [23,29,37] and architectures [36] for fast query processing, where no per-client states are maintained to avoid the corresponding overheads and to achieve the lowest latency. System model.…”

Section: Model and Definitionsmentioning

confidence: 99%

Distributed Transactional Systems Cannot Be Fast

Didona

Fatourou

Guerraoui

et al. 2019

The 31st ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

We prove that no fully transactional system can provide fast read transactions (including read-only ones that are considered the most frequent in practice). Specifically, to achieve fast read transactions, the system has to give up support of transactions that write more than one object. We prove this impossibility result for distributed storage systems that are causally consistent, i.e., they do not require to ensure any strong form of consistency. Therefore, our result holds also for any system that ensures a consistency level stronger than causal consistency, e.g., strict serializability. The impossibility result holds even for systems that store only two objects (and support at least two servers and at least four clients). It also holds for systems that are partially replicated. Our result justifies the design choices of state-of-the-art distributed transactional systems and insists that system designers should not put more effort to design fullyfunctional systems that support both fast read transactions and ensure causal or any stronger form of consistency.

show abstract

Section: Model and Definitionsmentioning

confidence: 99%

Distributed Transactional Systems Cannot Be Fast

Didona

Fatourou

Guerraoui

et al. 2019

The 31st ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

show abstract

“…This is because a server thread of RDMA-Memcached has to coordinate with other threads for sharing data structures (e.g, LRU lists) as well as to perform network operations, which does not exhibit good scalability [5,16,23]. By using data partition, a server thread in ServerReply does not need to interact with other threads, so ServerReply is only limited by outbound RDMA operations.…”

Section: Comparison On Throughputmentioning

confidence: 99%

RFP

Zhang

Kang

et al. 2017

Proceedings of the Twelfth European Conference on Computer Systems

View full text Add to dashboard Cite

Remote Direct Memory Access (RDMA) has been widely deployed in modern data centers. However, existing usages of RDMA lead to a dilemma between performance and redesign cost. They either directly replace socket-based send/receive primitives with the corresponding RDMA counterpart (server-reply), which only achieves moderate performance improvement; or push performance further by using one-sided RDMA operations to totally bypass the server (server-bypass), at the cost of redesigning the software.In this paper, we introduce two interesting observations about RDMA. First, RDMA has asymmetric performance characteristics, which can be used to improve server-reply's performance. Second, the performance of server-bypass is not as good as expected in many cases, because more rounds of RDMA may be needed if the server is totally bypassed. We therefore introduce a new RDMA paradigm called Remote Fetching Paradigm (RFP). Although RFP requires users to set several parameters to achieve the best performance, it supports the legacy RPC interfaces and hence avoids the need of redesigning application-specific data structures. Moreover, with proper parameters, it can achieve even higher IOPS than that of the previous paradigms.We have designed and implemented an in-memory keyvalue store based on RFP to evaluate its effectiveness. Experimental results show that RFP improves performance by 1.6⇥⇠4⇥ compared with both server-reply and serverbypass paradigms. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and /or a fee. Request permissions from permissions@acm.org. Figure 1. Improvements of RFP over server-reply and server-bypass. A higher bar means better performance.

show abstract

“…KVS such as Memcached [2], Redis [3], Dynamo [22], TAO [11], and Voldemort [5] are used in production environments of large service providers such as Facebook, Amazon, Twitter, Zynga, and LinkedIn [6,39,46,59]. The popularity of these systems has resulted in considerable research and development efforts, including open-source implementations [1], research prototypes [7,51] and a wide range of sophisticated, highly tuned frameworks that aspire to become the state-of-the-art of KVS [23,36,37].…”

Section: In-memory Key-value Storesmentioning

confidence: 99%

“…α = 0.99 is the typical data popularity distribution used in KVS research [16,23,31,36,37]. Some studies show that the popularity distribution skew in real-world datasets can be lower than that (e.g., α = 0.6 [55], α = 0.7 − 0.9 [8,53]), but also even higher (e.g., up to α = 1.01 [15,27]).…”

Section: Skew In Scale-out Architecturesmentioning

confidence: 99%

“…In CREW, the memory is managed as a single read-only pool, with changes being handled by a specific thread based on the location of the object in memory. Recent work has demonstrated the scalability benefits of the CREW model on Xeon-class servers [36,37]. As most workloads are read-dominated [9,11,16,52], CREW offers a sweet spot in terms of scalable performance by keeping synchronization requirements to a minimum.…”

Section: Concurrency Modelmentioning

confidence: 99%

See 1 more Smart Citation

The Case for RackOut

Novaković¹,

Daglis²,

Bugnion³

et al. 2016

Proceedings of the Seventh ACM Symposium on Cloud Computing

View full text Add to dashboard Cite

To provide low latency and high throughput guarantees, most large key-value stores keep the data in the memory of many servers. Despite the natural parallelism across lookups, the load imbalance, introduced by heavy skew in the popularity distribution of keys, limits performance. To avoid violating tail latency service-level objectives, systems tend to keep server utilization low and organize the data in micro-shards, which provides units of migration and replication for the purpose of load balancing. These techniques reduce the skew, but incur additional monitoring, data replication and consistency maintenance overheads.In this work, we introduce RackOut, a memory pooling technique that leverages the one-sided remote read primitive of emerging rack-scale systems to mitigate load imbalance while respecting service-level objectives. In RackOut, the data is aggregated at rack-scale granularity, with all of the participating servers in the rack jointly servicing all of the rack's micro-shards. We develop a queuing model to evaluate the impact of RackOut at the datacenter scale. In addition, we implement a RackOut proof-of-concept key-value store, evaluate it on two experimental platforms based on RDMA and Scale-Out NUMA, and use these results to validate the model. Our results show that RackOut can increase throughput up to 6× for RDMA and 8.6× for Scale-Out NUMA compared to a scale-out deployment, while respecting tight tail latency service-level objectives.

show abstract

Architecting to achieve a billion requests per second throughput on a single key-value store server platform

Cited by 16 publications

References 26 publications

Distributed Transactional Systems Cannot Be Fast

Distributed Transactional Systems Cannot Be Fast

RFP

The Case for RackOut

Contact Info

Product

Resources

About