Cats

Abstract

Distributed key-value stores provide scalable, faulttolerant, and self-organizing storage services, but fall short of guaranteeing linearizable consistency in partially synchronous, lossy, partitionable, and dynamic networks, when data is distributed and replicated automatically by the principle of consistent hashing [14]. This work introduces consistent quorums as a solution for achieving atomic consistency. We present the design and implementation of CATS, a key-value store which uses consistent quorums to guarantee linearizability and partition tolerance in such adverse and dynamic network conditions. CATS is scalable, elastic, and self-organizing; key properties for modern cloud storage middleware. Our system evaluation shows that consistency can be achieved with practical performance and modest overhead: 5% decrease in throughput for read-intensive workloads, and 25% throughput loss for write-intensive workloads. CATS delivers submillisecond operation latencies under light load, single-digit millisecond operation latencies at 50% load, and it sustains a throughput of one thousand operations per second, per server, while scaling linearly to hundreds of servers.Distributed key-value stores, such as Cassandra [15] and Dynamo [8], employ principles from DHTs [21] to build scalable and self-managing data stores. In contrast to CATS, these systems chose availability over atomic consistency, hence only providing eventual consistency [22]. While eventual consistency is sufficient for some applications, the complexities of merging divergent replicas can be non-trivial. We avoid the complexities entailed by eventual consistency while providing scalable storage for critical applications which need atomic consistency, guaranteeing it at the cost of a modest decrease in throughput. To handle dynamic networks, atomic registers were extended by protocols such as RAMBO [17], RAMBO II [9], RDS [7] and DynaStore [1] to be reconfigurable. Similarly, SMART [16] enabled reconfiguration in replicated state machines.With consistent quorums we provide high-throughput read/write operations without paying the full cost of state machine replication which needs coordination for every operation. Moreover, our design does not depend on electing a single leader and the complexities that come with that [5]. While these systems can handle dynamism and provide atomic consistency, they are not scalable as they were not designed to partition the data across a large number of machines. The novelty of CATS is in extending the reconfiguration techniques contributed by these works, such that they can be used at large scale, in order to build a system that is completely decentralized and self-managing.Distributed coordination systems such as Chubby [4] and ZooKeeper [11,13], provide linearizability and crashrecovery, but are not scalable. Master-based key-value stores, such as Bigtable [6], HBase [12], and MongoDB [18], rely on a central server for coordination and data partitioning. Similarly, Spinnaker [19] uses Zookeeper [11]. Since these system...