Reliability of flat XOR-based erasure codes on heterogeneous devices

Greenan, Kevin M.; Miller, Ethan L.; Wylie, Jay J.

doi:10.1109/dsn.2008.4630083

Cited by 29 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data placement issue has been considered in [2]. The emphasis of that work was on redundancy placement, namely, the placement of erasure coded data, rather than replica placement.…”

Section: Related Workmentioning

confidence: 99%

Effect of Replica Placement on the Reliability of Large-Scale Data Storage Systems

Venkatesan

Iliadis

et al. 2010

2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems

View full text Add to dashboard Cite

Abstract-Replication is a widely used method to protect largescale data storage systems from data loss when storage nodes fail. It is well known that the placement of replicas of the different data blocks across the nodes affects the time to rebuild. Several systems described in the literature are designed based on the premise that minimizing the rebuild times maximizes the system reliability. Our results however indicate that the reliability is essentially unaffected by the replica placement scheme. We show that, for a replication factor of two, all possible placement schemes have mean times to data loss (MTTDLs) within a factor of two for practical values of the failure rate, storage capacity, and rebuild bandwidth of a storage node. The theoretical results are confirmed by means of event-driven simulation. For higher replication factors, an analytical derivation of MTTDL becomes intractable for a general placement scheme. We therefore use one of the alternate measures of reliability that have been proposed in the literature, namely, the probability of data loss during rebuild in the critical mode of the system. Whereas for a replication factor of two this measure can be directly translated into MTTDL, it is only speculative of the MTTDL behavior for higher replication factors. This measure of reliability is shown to lie within a factor of two for all possible placement schemes and any replication factor. We also show that for any replication factor, the clustered placement scheme has the lowest probability of data loss during rebuild in critical mode among all possible placement schemes, whereas the declustered placement scheme has the highest probability. Simulation results reveal however that these properties do not hold for the corresponding MTTDLs for a replication factor greater than two. This indicates that some alternate measures of reliability may not be appropriate for comparing the MTTDL of different placement schemes.

show abstract

“…Data placement issue has been considered in [2]. The emphasis of that work was on redundancy placement, namely, the placement of erasure coded data, rather than replica placement.…”

Section: Related Workmentioning

confidence: 99%

Effect of Replica Placement on the Reliability of Large-Scale Data Storage Systems

Venkatesan

Iliadis

et al. 2010

2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems

View full text Add to dashboard Cite

show abstract

“…Greenan et al [14] show that this task can be complex for even relatively simple systems, and, in order to get the precise reliability of a storage system, some simulation methods must be used. With simulation, however, the precise reliability of a system can't be expressed in a closedform expression; rather, the simulation must be run for each point in time.…”

Section: Reliability Transition Functionmentioning

confidence: 99%

Understanding data survivability in archival storage systems

Miller

Long

2012

Proceedings of the 5th Annual International Systems and Storage Conference

Self Cite

View full text Add to dashboard Cite

Preserving data for a long period of time in the face of faults, large and small, is crucial for designing reliable archival storage systems. However, the survivability of data is different from the reliability of storage because typically, data are stored in more than one storage at a given moment. Previous studies of reliability ignore the former. We present a framework for relating data survivability and storage reliability, and use the framework to gauge the impact of rare but large-scale events on data survivability. We also present a method to track all copies of data and the condition of all the online and offline media, devices and systems on which they are stored uninterruptedly over the whole lifetime of the data. With this method, the survivability of the data can be closely monitored, and potential dangers can be handled in a timely manner. A better understanding of data survivability can be used in reducing unnecessary data replicas, thus reducing the cost.

show abstract

“…Especially when applied to those storage systems that are composed of heterogeneous devices with different failure and recovery rates, GRID codes can be further exploited to work better by using the method proposed in Greenan and Wylie [2008]. …”

Section: High Fault Tolerancementioning

confidence: 99%

GRID codes

Shu

Zheng

2009

ACM Trans. Storage

View full text Add to dashboard Cite

As storage systems grow in size and complexity, they are increasingly confronted with concurrent disk failures together with multiple unrecoverable sector errors. To ensure high data reliability and availability, erasure codes with high fault tolerance are required. In this article, we present a new family of erasure codes with high fault tolerance, named GRID codes. They are called such because they are a family of strip-based codes whose strips are arranged into multi-dimensional grids. In the construction of GRID codes, we first introduce a concept of matched codes and then discuss how to use matched codes to construct GRID codes. In addition, we propose an iterative reconstruction algorithm for GRID codes. We also discuss some important features of GRID codes. Finally, we compare GRID codes with several categories of existing codes. Our comparisons show that for large-scale storage systems, our GRID codes have attractive advantages over many existing erasure codes: (a) They are completely XOR-based and have very regular structures, ensuring easy implementation; (b) they can provide up to 15 and even higher fault tolerance; and (c) their storage efficiency can reach up to 80% and even higher. All the advantages make GRID codes more suitable for large-scale storage systems. , W. 2009. GRID codes: Strip-based erasure codes with high fault tolerance for storage systems.

show abstract

Reliability of flat XOR-based erasure codes on heterogeneous devices

Cited by 29 publications

References 16 publications

Effect of Replica Placement on the Reliability of Large-Scale Data Storage Systems

Effect of Replica Placement on the Reliability of Large-Scale Data Storage Systems

Understanding data survivability in archival storage systems

GRID codes

Contact Info

Product

Resources

About