Erasure-Coded Byzantine Storage with Separate Metadata

Abstract

Although many distributed storage protocols have been introduced, a solution that combines the strongest properties in terms of availability, consistency, fault-tolerance, storage complexity and the supported level of concurrency, has been elusive for a long time. Combining these properties is difficult, especially if the resulting solution is required to be efficient and incur low cost.We present AWE, the first erasure-coded distributed implementation of a multi-writer multireader read/write storage object that is, at the same time: (1) asynchronous, (2) wait-free, (3) atomic, (4) amnesic, (i.e., with data nodes storing a bounded number of values) and (5) Byzantine faulttolerant (BFT) using the optimal number of nodes. Furthermore, AWE is efficient since it does not use public-key cryptography and requires data nodes that support only reads and writes, further reducing the cost of deployment and ownership of a distributed storage solution. Notably, AWE stores metadata separately from k-out-of-n erasure-coded fragments. This enables AWE to be the first BFT protocol that uses as few as 2t + k data nodes to tolerate t Byzantine nodes, for any k ≥ 1.

A coded shared atomic memory algorithm for message passing architectures

“…Among the previous works, [7,10,14,15,20,21] have similar correctness requirements as our paper; these references aim to emulate an atomic shared memory that supports concurrent operations in asynchronous networks. We compare our algorithms with the ORCAS-A and ORCAS-B algorithms of [15], the algorithm of [20], which we call the GWGR algorithm, the algorithm of [21], which we call the HGR algorithm, the M-PoWerStore algorithm of [14], the algorithm of [10], which we call the CT algorithm, and the AWE algorithm of [7]. We note that [15] assumes lossy channels and [10,14,21] assume Byzantine failures.…”

“…As noted in the table, a distinguishing feature of CASGC is that it simultaneously has small worst-case communication cost, a bounded storage cost and desirable liveness properties when we consider the class of executions where the number of ongoing write operations, message delays and the rate of client failures are bounded. Here, we make some remarks comparing the storage costs, liveness properties and communication costs of our algorithms with the algorithms of [7,10,14,15,20,21]. …”

“…Erasure coding has been used to develop shared memory emulation techniques for systems with crash failures in [3,4,15,33] and Byzantine failures in [2,7,10,14,20,21]. In erasure coding, note that each server stores a coded element, so a reader has to obtain enough coded elements to decode and return the value.…”

“…Since the late 1970s, emulation of shared-memory systems in distributed message-passing environments has been an active area of research [2][3][4][5][6][7][8]10,[14][15][16][17][18][19][20][21]28,33,34]. The traditional approach to building redundancy for distributed systems in the context of shared memory emulation is replication.…”

“…3). Algorithms for shared memory emulation that use the idea of erasure coding to store multiple versions of a data object consistently have been developed in [2][3][4]6,7,10,14,15,20,21,33]. In this paper, we develop algorithms that improve on previous algorithms in terms of communication and storage costs.…”