A data-assisted reliability model for carrier-assisted cold data storage systems

Abstract

Cold data storage systems are used to allow long term digital preservation for institutions' archive. The common functionality among cold and warm/hot data storage is that the data is stored on some physical medium for read-back at a later time. However in cold storage, write and read operations are not necessarily done in the same exact geographical location. Hence, a third party assistance is typically utilized to bring together the medium and the drive. On the other hand, the reliability modeling of such a decomposed system poses few challenges that do not necessarily exist in other warm/hot storage alternatives such as fault detection and absence of the carrier, all totaling up to the data unavailability issues. In this paper, we propose a generalized non-homogenous Markov model that encompasses the aging of the carriers in order to address the requirements of today's cold data storage systems in which the data is encoded and spread across multiple nodes for the long-term data retention. We have derived useful lower/upper bounds on the overall system availability. Furthermore, the collected field data is used to estimate parameters of a Weibull distribution to accurately predict the lifetime of the carriers in an example scale-out setting. In this study, we numerically demonstrate the significance of carriers' presence and the key role that their timely maintenance plays on the long-term reliability and availability of the stored content.

Durability and Availability of Erasure-Coded Storage Systems with Concurrent Maintenance

“…Therefore, this is a solid use case where the foundations that we laid out in previous chapters about the reliability and availability. The contents of this section is largely covered in [43].…”

Cloud2HDD: Large-Scale HDD Data Analysis on Cloud for Cloud Datacenters

“…Many techniques have been proposed in lietrature to improve cloud storage systems against such failures such as information fragmentation [2], and reinforcement learning [3]. Recently, data modeling attempts are made in order to incorporate more drive-related parameters and failure data in the durability prediction of arrays of hard drives [4], [5]. On the other hand, the methods and technology used by manufacturers during the building process can create a faulty connection between distinct storage devices.…”

“…One of most important is the hardness of the prediction model which has to take into account the correlated error scenarios as well as the usage and workload pattern. This problem can be relieved by studying the accumulated data over a time period in which the correlation of failures would be captured by the collected data itself [5]. However as the volume of data exceeds the available resources of a single machine, it became mandatory to analyze data on the cloud (computing) for the cloud (storage) systems such as big data centers [6].…”