A new reliability model in replication-based big data storage systems

Wang, Jun; Wu, Huafeng; Wang, Ruijun

doi:10.1016/j.jpdc.2017.02.001

Cited by 22 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The concept of replication is used to ensure these requirements. The main idea of replication is to manage large volumes of data in a distributed manner, speeds up data access, reduces access latency and increases data availability [4], [5], [6].…”

Section: Reactive Fault Tolerancementioning

confidence: 99%

A Review on Fault Tolerance in Distributed Database

2021

IJATCSE

View full text Add to dashboard Cite

In this paper, we study about the different types of fault tolerance techniques which are used in various distributed database systems. The main focus of this research is about how the data are storedin the servers, fault detection techniques and the recovery techniques used. A fault can occur for many reasons. For example, system failure, resource failure, network between the server’s failure and any other reasons. These faults must be emphasis in order to make sure the system can work smoothly without any problem. A proper failure detector and a reliable fault tolerance technique can avoid loss and at once save the system from fail.

show abstract

Section: Reactive Fault Tolerancementioning

confidence: 99%

A Review on Fault Tolerance in Distributed Database

2021

IJATCSE

View full text Add to dashboard Cite

show abstract

“…In that work, they demonstrated that two replicas were enough for protecting data loss under independent node failure while three replicas were enough to protect data under correlated failure and stored the first two replicas on primary tier and the third replica on backup tier without effecting the performance. In [10], Wang et al proposed a comprehensive re-liability model that considered not only probability of data loss but also bandwidth allocation in the recovery process. They used proposed reliability model for analyzing reliability and system repair rate for different data layout schemes, namely copyset replication, shifted declustering and random declustering layouts.…”

Section: Related Workmentioning

confidence: 99%

Avoiding Performance Impacts by Re-Replication Workload Shifting in HDFS Based Cloud Storage

Shwe

Aritsugi

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Data replication in cloud storage systems brings a lot of benefits, such as fault tolerance, data availability, data locality and load balancing both from reliability and performance perspectives. However, each time a datanode fails, data blocks stored on the failed datanode must be restored to maintain replication level. This may be a large burden for the system in which resources are highly utilized with users' application workloads. Although there have been many proposals for replication, the approach of re-replication has not been properly addressed yet. In this paper, we present a deferred re-replication algorithm to dynamically shift the re-replication workload based on current resource utilization status of the system. As workload pattern varies depending on the time of the day, simulation results from synthetic workload demonstrate a large opportunity for minimizing impacts on users' application workloads with the simple algorithm that adjusts re-replication based on current resource utilization. Our approach can reduce performance impacts on users' application workloads while ensuring the same reliability level as default HDFS can provide.

show abstract

“…Wang et al [19] proposed a reliability model that took into consideration both data loss probability and bandwidth allocation for recovery process. Then, they analyzed reliability and system repair rate for different data layout schemes.…”

Section: Related Workmentioning

confidence: 99%

A Data Re-Replication Scheme and Its Improvement Toward Proactive Approach

Shwe

Aritsugi

2018

AEJ

View full text Add to dashboard Cite

With increasing demand for cloud computing technology, cloud infrastructures are utilized to their maximum limits. There is a high possibility that commodity servers that are used in Hadoop Distributed File System (HDFS) based cloud data center will fail often. However, the selection of source and destination data nodes for re-replication of data has so far not been adequately addressed. In order to balance the workload among nodes during re-replication phase and reduce impact on cluster normal jobs’ performance, we develop a re-replication scheme that takes into consideration of both performance and reliability perspectives. The appropriate nodes for re-replication are selected based on Analytic Hierarchy Process (AHP) with the consideration of the current utilization of resources by the cluster normal jobs. Toward effective data re-replication, we investigate the feasibility of using linear regression and local regression methods to predict resource utilization. Simulation results show that our proposed approach can reduce re-replication time, total job execution time and top-of-rack network traffic compared to baseline HDFS, consequently increases the reliability of the system and reduces performance impacts on users jobs. Regarding feasibility study of prediction methods, both regression methods are good enough to predict short time future resource utilization for re-replication.

show abstract

A new reliability model in replication-based big data storage systems

Cited by 22 publications

References 12 publications

A Review on Fault Tolerance in Distributed Database

A Review on Fault Tolerance in Distributed Database

Avoiding Performance Impacts by Re-Replication Workload Shifting in HDFS Based Cloud Storage

A Data Re-Replication Scheme and Its Improvement Toward Proactive Approach

Contact Info

Product

Resources

About