Early Identification of Critical Blocks: Making Replicated Distributed Storage Systems Reliable Against Node Failures

Fang, Juntao; Wan, Shenggang; Huang, Ping; Xie, Changsheng; He, Xubin

doi:10.1109/tpds.2018.2833457

Cited by 8 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [20] directed significant research attention toward early identification of node failures in distributed systems, which consist of hundreds and thousands of nodes. However, identification only considers identification time capability to ac-celerate the maintaining process in distributed storage systems.…”

Section: Related Workmentioning

confidence: 99%

Enhanced faulty node detection with interval weighting factor for distributed systems

2021

View full text Add to dashboard Cite

This paper proposes an enhanced faulty node detection method using interval weighting factor, which monitors node behavior using pseudo-random Bose-Chaudhuri-Hocquenghem (BCH) code for distributed networked control systems. Master node collects the replacement of the cyclic redundancy check (CRC) codes by a single-bit BCH code of each slave node. However, BCH code can only obtain error position of the suspected faulty node without consideration of channel nodes. Hence, suspected error nodes are saved within the detected error interval and normalized using the weighting factor, which is carried out during sequential check, for interpretation. Fault judgement is carried out to adequately interpret the data, to guarantee detection accuracy of the detected error. The data is represented by statistical characteristic of the raw data and filtered data. This scheme can be applied to detect and prevent the severe damage by node failure. The simulation results prove the effectiveness of the interval faulty weighting factor, in obtaining raw data from the monitoring of the BCH code and filtered data, which are more accurate representation than the raw data. Moreover, the characteristics of the observed data were verified as the evaluation result of the suspected faulty node.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhanced faulty node detection with interval weighting factor for distributed systems

2021

View full text Add to dashboard Cite

show abstract

Scalable local reconstruction code design for hot data reads in cloud storage systems

Zhang

2022

Sci. China Inf. Sci.

View full text Add to dashboard Cite

ACPR: Adaptive Classification Predictive Repair Method for Different Fault Scenarios

Song,

Zheng,

Tian

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Erasure codes are widely used in large-scale distributed storage systems due to their high efficiency and reliability, but they also face extremely high repair penalties when data corruption occurs. At present, machine learning methods can accurately predict the next failure time and type of machine nodes. Based on this, in order to solve the problem of unnecessary repair traffic caused by temporary failures, as well as the more degraded reads of high-frequency accessed data due to longer failure time of such data in existing repair methods, we propose an Adaptive Classification Predictive Repair method (ACPR) for different fault scenarios. By categorizing the failed blocks into high-risk and low-risk based on the failure type of the soon-to-fail (STF) node and the access heat of STF blocks, ACPR can perform adaptive predictive repair. By quickly repair high-risk blocks to ensure data availability while delaying the repair of low-risk blocks, a large amount of unnecessary repair traffic caused by temporary node failures in the cluster is avoided. Alibaba Cloud Elastic Compute Service (ECS) experiments results show that compared with FastPR and ECPipe, ACPR can shorten the repair time per data block by up to 15.2% and 33.5%, respectively. Moreover, ACPR can reduce repair traffic by up to 74.1% and 84.4%, respectively.INDEX TERMS Distributed storage system, data recovery, erasure coding.

show abstract

Early Identification of Critical Blocks: Making Replicated Distributed Storage Systems Reliable Against Node Failures

Cited by 8 publications

References 19 publications

Enhanced faulty node detection with interval weighting factor for distributed systems

Enhanced faulty node detection with interval weighting factor for distributed systems

Scalable local reconstruction code design for hot data reads in cloud storage systems

ACPR: Adaptive Classification Predictive Repair Method for Different Fault Scenarios

Contact Info

Product

Resources

About