Coding-based replication schemes for distributed systems

Abstract

Data is often replicated in distributed systems to improve availability and performance. This replication is expensive in terms of disk storage since the existing schemes generally require full files to be stored at each site. In this paper, we present schemes which significantly reduce the storage requirements in replication based systems. These schemes use the coding method suggested by Rabin to store replicated data. The first scheme that we present is a modification of the simple voting algorithm and its quorum requirements. , We then show how some of the extensions of the voting algorithm can also be modified to get storage efficient schemes for managing such replication. We evaluate the availability offered by these schemes and show that the storage space required to achieve certain availability are signiscanfly lower than the conventional schemes with full file replication. Since coding is used, these schemes also provide a high degree of data security.

A coded shared atomic memory algorithm for message passing architectures

“…We present a brief introduction of the technique of erasure coding in Sect. 3. For a storage system with N nodes, we show in Theorem 3 that CAS ensures the following liveness property: all operations that are invoked by a non-failed client terminate provided that the number of server failures is bounded by a parameter f, where f < N 2 and regardless of the number of client failures.…”

“…During the write operation, the storage cost of implementing the message dispersal primitive during an operation can be as large as the storage cost of replication. The storage and garbage 3 As we shall see later, the server gossip is not essential to correctness of CAS. It is however useful as a theoretical tool to prove correctness of CASGC.…”

“…Erasure coding has been used to develop shared memory emulation techniques for systems with crash failures in [3,4,15,33] and Byzantine failures in [2,7,10,14,20,21]. In erasure coding, note that each server stores a coded element, so a reader has to obtain enough coded elements to decode and return the value.…”

“…Since the late 1970s, emulation of shared-memory systems in distributed message-passing environments has been an active area of research [2][3][4][5][6][7][8]10,[14][15][16][17][18][19][20][21]28,33,34]. The traditional approach to building redundancy for distributed systems in the context of shared memory emulation is replication.…”

“…3). Algorithms for shared memory emulation that use the idea of erasure coding to store multiple versions of a data object consistently have been developed in [2][3][4]6,7,10,14,15,20,21,33]. In this paper, we develop algorithms that improve on previous algorithms in terms of communication and storage costs.…”