Separating Data and Control: Asynchronous BFT Storage with 2t + 1 Data Replicas

Abstract

The cost of Byzantine Fault Tolerant (BFT) storage is the main concern preventing its adoption in practice. This cost stems from the need to maintain at least 3t + 1 replicas in different storage servers in the asynchronous model, so that t Byzantine replica faults can be tolerated.In this paper, we present MDStore, the first fully asynchronous read/write BFT storage protocol that reduces the number of data replicas to as few as 2t + 1, maintaining 3t + 1 replicas of metadata at (possibly) different servers. At the heart of MDStore store is its metadata service that is built upon a new abstraction we call timestamped storage. Timestamped storage both allows for conditional writes (facilitating the implementation of a metadata service) and has consensus number one (making it implementable wait-free in an asynchronous system despite faults). In addition to its low data replication factor, MDStore offers very strong guarantees implementing multi-writer multi-reader atomic wait-free semantics and tolerating any number of Byzantine readers and crash-faulty writers.We further show that MDStore data replication overhead is optimal; namely, we prove a lower bound of 2t + 1 on the number of data replicas that applies even to crash-tolerant storage with a fault-free metadata service oracle. Finally, we prove that separating data from metadata for reducing the cost of BFT storage is not possible without cryptographic assumptions. However, our MDStore protocol uses only lightweight cryptographic hash functions.

Hardening Cassandra Against Byzantine Failures

“…By combining the above 3 requirements, we obtain: The last bound was formally proved by Malkhi & Reiter [41]. Cachin et al [13] have lowered this bound to 2f + 1 by separating between the actual data and its metadata; storing the medadata still requires 3f + 1 nodes. The above separation was presented under the assumptions of benign writers and Byzantine readers.…”

“…However, ever since the seminal PBFT work of Castro and Liskov [14], the practicality of building Byzantine fault tolerant replicated state machines has been demonstrated by multiple academic projects, e.g., [16,28] to name a few. Interestingly, storage systems offer weaker semantics than general replicated state machines, and therefore it may be possible to make them resilient to Byzantine failures using weaker timing and failure detection assumptions, as been proposed in [13,38,41,42]. Yet, to the best of our knowledge, we are the first to offer an extension of Cassandra that can withstand Byzantine failures.…”

BFT in Blockchains: From Protocols to Use Cases

“…The two virtual machines monitor each other so that even one fails, the other one can still guarantee that only consistent messages are sent. Several previous eforts separate the roles in BFT for diferent purposes [31,56,102,215,223,229]. Wang et al [223], Cachin et al [31,56], Duan et al [102], and Yin et al [229] use an architecture that separates the agreement of client requests from the execution of the operations.…”

“…Several previous eforts separate the roles in BFT for diferent purposes [31,56,102,215,223,229]. Wang et al [223], Cachin et al [31,56], Duan et al [102], and Yin et al [229] use an architecture that separates the agreement of client requests from the execution of the operations. A simple architecture is illustrated in Fig.…”

Recent Results on Fault-Tolerant Consensus in Message-Passing Networks

“…The system requires 3f + 1 repli-cas. Recently, Cachin et al built a BFT storage system, MDStore, which only requires 2f + 1 replicas under the assumption that the client is always fault-free when writing data [33,34]. MDStore system had two novelties: (i) separation of data and metadata storage, and (ii) metadata service based on lightweight cryptographic hash functions.…”