Design and Evaluation of a Risk-Aware Failure Identification Scheme for Improved RAS in Erasure-Coded Data Centers

Huang, Weichen; Fang, Juntao; Wan, Shenggang; Xie, Changsheng; He, Xubin

doi:10.1109/tpds.2020.3010048

Cited by 3 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the method of delayed repair generally delays the repair of failed data by enhancing the reliability of some stripes. For example, Risk-Aware Failure Identification (RAFI) [29] and Lazy Repair with Temporary Redundancy (LRTR) [24], where RAFI enhances reliability by accelerating the repair of stripes with more failed blocks and delaying the repair of stripes with fewer failed blocks, reducing unnecessary repair traffic. LRTR performs low-cost temporary redundancy operations on the surviving blocks of the affected stripes and delays the repair of all failed data, reducing unnecessary repair traffic.…”

Section: Related Workmentioning

confidence: 99%

ACPR: Adaptive Classification Predictive Repair Method for Different Fault Scenarios

Song,

Zheng,

Tian

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Erasure codes are widely used in large-scale distributed storage systems due to their high efficiency and reliability, but they also face extremely high repair penalties when data corruption occurs. At present, machine learning methods can accurately predict the next failure time and type of machine nodes. Based on this, in order to solve the problem of unnecessary repair traffic caused by temporary failures, as well as the more degraded reads of high-frequency accessed data due to longer failure time of such data in existing repair methods, we propose an Adaptive Classification Predictive Repair method (ACPR) for different fault scenarios. By categorizing the failed blocks into high-risk and low-risk based on the failure type of the soon-to-fail (STF) node and the access heat of STF blocks, ACPR can perform adaptive predictive repair. By quickly repair high-risk blocks to ensure data availability while delaying the repair of low-risk blocks, a large amount of unnecessary repair traffic caused by temporary node failures in the cluster is avoided. Alibaba Cloud Elastic Compute Service (ECS) experiments results show that compared with FastPR and ECPipe, ACPR can shorten the repair time per data block by up to 15.2% and 33.5%, respectively. Moreover, ACPR can reduce repair traffic by up to 74.1% and 84.4%, respectively.INDEX TERMS Distributed storage system, data recovery, erasure coding.

show abstract

Section: Related Workmentioning

confidence: 99%

ACPR: Adaptive Classification Predictive Repair Method for Different Fault Scenarios

Song,

Zheng,

Tian

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Each part of the equipment is scattered, and some types of server equipment have a large number and scattered locations. erefore, WAN connection is adopted between different server devices [19].…”

Section: Build a Big Data Classification Platform Withmentioning

confidence: 99%

Intelligent Classification Method of Low Occupancy Big Data Based on Grid Index

Zhao

2021

Scientific Programming

View full text Add to dashboard Cite

In order to enhance the load balance in the big data storage process and improve the storage efficiency, an intelligent classification method of low occupancy big data based on grid index is studied. A low occupancy big data classification platform was built, the infrastructure layer was designed using grid technology, grid basic services were provided through grid system management nodes and grid public service nodes, and grid application services were provided using local resource servers and enterprise grid application services. Based on each server node in the infrastructure layer, the basic management layer provides load forecasting, image backup, and other functional services. The application interface layer includes the interfaces required for the connection between the platform and each server node, and the advanced access layer provides the human-computer interaction interface for the operation of the platform. Finally, based on the obtained main structure, the depth confidence network is constructed by stacking several RBM layers, the new samples are expanded by adding adjacent values to obtain the mean value, and the depth confidence network is used to classify them. The experimental results show that the load of different virtual machines in the low occupancy big data storage process is less than 40%, and the load of each virtual machine is basically the same, indicating that this method can enhance the load balance in the data storage process and improve the storage efficiency.

show abstract