Durability and Availability of Erasure-Coded Storage Systems with Concurrent Maintenance

Abstract

This initial version of this document was written back in 2014 for the sole purpose of providing fundamentals of reliability theory as well as to identify the theoretical types of machinery for the prediction of durability/availability of erasure-coded storage systems. Since the definition of a "system" is too broad, we specifically focus on warm and cold storage systems where the data is stored in a distributed fashion across different storage units with or without continuous (full duty-cycle) operation.The contents of this document are dedicated to a review of fundamentals, a few major improved stochastic models, and several contributions of my work relevant to the field. One of the interesting contributions of this document is the introduction of the most general form of Markov models for the estimation of mean time to failure numbers. This work was partially later published in IEEE Transactions on Reliability. Very good approximations for the closed-form solutions for this general model are also investigated. Various storage configurations under different policies are compared using such advanced models. Later in a subsequent chapter, we have also considered multi-dimensional Markov models to address detached drive-medium combinations such as those found in optical disk and tape storage systems. It is not hard to anticipate such a system structure would most likely be part of future DNA storage libraries and hence find a plethora of interesting applications. This work is partially published in Elsevier Reliability and System Safety.Topics that include simulation modelings for more accurate estimations are included towards the end of the document by noting the deficiencies of the simplified canonical as well as more complex Markov models, due mainly to the stationary and static nature of Markovinity. Throughout the document, we shall focus on concurrently maintained systems although the discussions will only slightly change for the systems repaired one device at a time. The document is still under construction and future versions might likely include newer models and novel approaches to enrich the present contents. Some background on probability and coding theory might be expected that are briefly mentioned in the beginning of the document.