Reliability of Clustered vs. Declustered Replica Placement in Data Storage Systems

Abstract

Abstract-The placement of replicas across storage nodes in a replication-based storage system is known to affect rebuild times and therefore system reliability. Earlier work has shown that, for a replication factor of two, the reliability is essentially unaffected by the replica placement scheme because all placement schemes have mean times to data loss (MTTDLs) within a factor of two for practical values of the failure rate, storage capacity, and rebuild bandwidth of a storage node. However, for higher replication factors, simulation results reveal that this no longer holds. Moreover, an analytical derivation of MTTDL becomes intractable for general placement schemes. In this paper, we develop a theoretical model that is applicable for any replication factor and provides a good approximation of the MTTDL for small failure rates. This model characterizes the system behavior by using an analytically tractable measure of reliability: the probability of the shortest path to data loss following the first node failure. It is shown that, for highly reliable systems, this measure approximates well the probability of all paths to data loss after the first node failure and prior to the completion of rebuild, and leads to a rough estimation of the MTTDL. The results obtained are of theoretical and practical importance and are confirmed by means of simulations. As our results show, the declustered placement scheme, contrary to intuition, offers a reliability for replication factors greater than two that does not decrease as the number of nodes in the system increases.

Reliability of Geo-replicated Cloud Storage Systems

“…The reliability benefits of using declustered placement along with an intelligent rebuild scheme, which prioritizes the rebuild of the most critical data, were studied in [8]. However, to the best of our knowledge, the reliability of geo-replicated deployments with different network bandwidths within and between sites, has not been studied to date.…”

“…In the area of reliability analysis, a theoretical framework based on the direct path approach has been proposed to study the effect of data placement, redundancy schemes, latent sector errors, failure and rebuild time distributions, and network bandwidth on the reliability of storage systems [8], [9], [17], [18]. In particular, it has been shown that limited network bandwidth can significantly affect rebuild processes for certain data placement schemes [9].…”

“…The corresponding effective rebuild bandwidth B eff is therefore given by B eff = min( [1 + (d/2)] b d , B D ) . As the read and write rebuild operations are performed in parallel on different disks, the corresponding effective rebuild bandwidth b for each of the (d/2) + 1 replication sources (disks) is equal to b = B eff /[(d/2) + 1], given by (8). The remaining (1 − α 1 ) c/d data will be recovered from d 3 using the disk rebuild bandwidth b d , since the rebuild from the remaining d/2 replication sources has been completed.…”

“…Besides staged rebuild, another way to improve reliability is to use an intelligent rebuild scheme [4], [8], [9]. This scheme keeps track of the number of replicas of data in the system and gives priority to rebuilds of the data with the least number of remaining replicas.…”

Analytical Investigation of Availability in a Vision Cloud Storage Cluster

“…In this context, the majority of the work mainly considers replica placement policies [12,19] without taking into consideration real case studies as done in our work. In fact, our model could be exploited by a VISION Cloud administrator in order to opportunely build the infrastructure accordingly to the desired availability level both from the hardware (e.g., computation, storage, network resources) and the software (e.g., replication schema, cluster file system configuration) points of view.…”