Predicting Hard Drive Failures for Cloud Storage Systems

Liu, Dongshi; Wang, Bo; Li, Peng; Stones, Rebecca J.; Marbach, Trent G.; Wang, Gang; Liu, Xiaoguang; Li, Zhongwei

doi:10.1007/978-3-030-38991-8_25

Cited by 6 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies by Liu et al [ 6 ], Zang et al [ 49 ], and Santo et al [ 50 ], on the other hand, use a deep learning approach for hard drive failure prediction. The work of Liu et al focuses on cloud storage systems and applied modified recurrent neural networks (RNN).…”

Section: State Of the Artmentioning

confidence: 99%

“…The most common failures can be categorised as logical, mechanical, or firmware failures ( (accessed on 6 June 2021)). Such failures can result in system unavailability or even permanent data loss, which can have a negative impact due to system downtime and can lead to monetary losses for companies [ 6 ]. For example, 78% of hardware replacements in Microsoft ( (accessed on 10 June 2021)) data centres were due to HD failure [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Knowledge Graph Based Hard Drive Failure Prediction

Chhetri

Kurteva

Adigun

et al. 2022

Sensors

View full text Add to dashboard Cite

The hard drive is one of the important components of a computing system, and its failure can lead to both system failure and data loss. Therefore, the reliability of a hard drive is very important. Realising this importance, a number of studies have been conducted and many are still ongoing to improve hard drive failure prediction. Most of those studies rely solely on machine learning, and a few others on semantic technology. The studies based on machine learning, despite promising results, lack context-awareness such as how failures are related or what other factors, such as humidity, influence the failure of hard drives. Semantic technology, on the other hand, by means of ontologies and knowledge graphs (KGs), is able to provide the context-awareness that machine learning-based studies lack. However, the studies based on semantic technology lack the advantages of machine learning, such as the ability to learn a pattern and make predictions based on learned patterns. Therefore, in this paper, leveraging the benefits of both machine learning (ML) and semantic technology, we present our study, knowledge graph-based hard drive failure prediction. The experimental results demonstrate that our proposed method achieves higher accuracy in comparison to the current state of the art.

show abstract

Section: State Of the Artmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Knowledge Graph Based Hard Drive Failure Prediction

Chhetri

Kurteva

Adigun

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…data for the same SMART attribute can vary in meaning and values, depending on the hard drive model and the manufacturer [13], [22], [37], [38]. Therefore, to ensure that the behavior is homogeneous for all of the disks in the dataset, a single hard drive model is chosen.…”

Section: ) Benchmark Datasetsmentioning

confidence: 99%

“…In general, the approaches used in the literature are very different in nature, in the type of data they require, in the evaluation metrics they apply, etc. In this sense, carrying out a fair comparison is not always possible [4], [13]. Taking this into account, relevant and recent works that have also addressed the problem as a binary classification problem and are sufficiently detailed to be replicated are selected.…”

Section: ) Benchmark Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

A Multivariate Time Series Streaming Classifier for Predicting Hard Drive Failures [Application Notes]

Ircio

Lojo

Lozano

et al. 2022

IEEE Comput. Intell. Mag.

View full text Add to dashboard Cite

Digital data storage systems such as hard drives can suffer breakdowns that cause the loss of stored data. Due to the cost of data and the damage that its loss entails, hard drive failure prediction is vital. In this context, the objective of this paper is to develop a method for detecting the beginning of hard drive malfunction using streaming SMART data, allowing the user to take actions before the breakdown occurs. This is a challenging task for two main reasons. First, there are not usually many examples of failed hard drives. Second, in these few available examples, hard drives are only identified and labeled as failed after complete breakdown occurs, but the exact moment when they begin to malfunction is usually unknown. Both these aspects significantly complicate the supervised learning of hard drive failure prediction models. To cope with these issues, the problem is addressed as a multidimensional time series streaming classification problem based on sliding windows. Moreover, as a solution to the highly imbalanced situation, the learned classifier is optimized to maximize the minimum recall of classes. Experimental results using the Backblaze benchmark dataset show that the proposed method reliably anticipates hard drive failures and obtains a higher balance between the recall values of both classes, failed and correct disks, compared to other state-of-the-art solutions.

show abstract