DLFT: Data and Layout Aware Fault Tolerance Framework for Big Data Transfer Systems

Kasu, Preethika; Hamandawana, Prince; Chung, Tae-Sun

doi:10.1109/access.2021.3055731

Cited by 5 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section reviews the recently developed models for task scheduling with fault tolerance in CC. Kasu et al [11] presented a bloom filter enabled data-aware probability based fault tolerant (DAFT) method that might deal with these faults. Also, presented a data and layout aware method for fault tolerance (DLFT) to efficiently deal with the false positives match.…”

Section: Related Workmentioning

confidence: 99%

Gorilla Troops Optimizer Based Fault Tolerant Aware Scheduling Scheme for Cloud Environment

Muralidharan¹,

Latha²

2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

In cloud computing (CC), resources are allocated and offered to the clients transparently in an on-demand way. Failures can happen in CC environment and the cloud resources are adaptable to fluctuations in the performance delivery. Task execution failure becomes common in the CC environment. Therefore, fault-tolerant scheduling techniques in CC environment are essential for handling performance differences, resource fluxes, and failures. Recently, several intelligent scheduling approaches have been developed for scheduling tasks in CC with no consideration of fault tolerant characteristics. With this motivation, this study focuses on the design of Gorilla Troops Optimizer Based Fault Tolerant Aware Scheduling Scheme (GTO-FTASS) in CC environment. The proposed GTO-FTASS model aims to schedule the tasks and allocate resources by considering fault tolerance into account. The GTO-FTASS algorithm is based on the social intelligence nature of gorilla troops. Besides, the GTO-FTASS model derives a fitness function involving two parameters such as expected time of completion (ETC) and failure probability of executing a task. In addition, the presented fault detector can trace the failed tasks or VMs and then schedule heal submodule in sequence with a remedial or retrieval scheduling model. The experimental validation of the GTO-FTASS model has been performed and the results are inspected under several aspects. Extensive comparative analysis reported the better outcomes of the GTO-FTASS model over the recent approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Gorilla Troops Optimizer Based Fault Tolerant Aware Scheduling Scheme for Cloud Environment

Muralidharan¹,

Latha²

2023

Intelligent Automation &Amp; Soft Computing

View full text Add to dashboard Cite

show abstract

“…Kasu et al [28] proposed a DLBF data structure to avoid false-positive errors with object based big data transfer systems. To avoid false positives, object layout information (n-bits) is appended to the standard Bloom filter as an additional information about the object.…”

Section: Performance Optimization Of the Bloom Filtermentioning

confidence: 99%

“…However, as Bloom filters are probabilistic data structures, membership query result in false positives. This section presents a modified version of the Bloom filter, the DLBF [28], to handle false-positive errors more efficiently than a standard Bloom filter.…”

Section: Data-and Layout-aware Bloom Filter (Dlbf)mentioning

confidence: 99%

See 1 more Smart Citation

CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer Systems

2023

Self Cite

View full text Add to dashboard Cite

Various components are involved in the end-to-end path of data transfer. Protecting data integrity from failures in these intermediate components is a key feature of big data transfer tools. Although most of these components provide some degree of data integrity, they are either too expensive or inefficient in recovering corrupted data. This problem highlights the need for application-level end-to-end integrity verification during data transfer. However, the computational, memory, and storage overhead of big data transfer tools can be a significant bottleneck for ensuring data integrity due to the large size of the data. This paper proposes a novel framework for data integrity verification in big data transfer systems using a cross-referencing Bloom filter. This framework has three advantages over state-of-the-art data integrity techniques: lower computation and memory overhead and zero false-positive errors for a limited number of elements. This study evaluates the computation, memory, recovery time, and false-positive overhead for the proposed framework and compares them with state-of-the-art solutions. The evaluation results indicate that the proposed framework is efficient in detecting and recovering from integrity errors while eliminating false positives in the Bloom filter data structure. In addition, we observe negligible computation, memory, and recovery overheads for all workloads.

show abstract

“…These redundant data can directly or indirectly generate the original data and solve the problem of data transmission or storage. Loss problem, in order to ensure the stable operation of the system [1][2].…”

Section: Introductionmentioning

confidence: 99%

Performance Optimization Technology of Fault Tolerance Mechanism in Distributed System Based on Neural Network

2022

DPS

View full text Add to dashboard Cite

In the era of big data, distributed systems(DS) have to deal with more and more data, and the possibility of system hardware failure is higher and higher. Traditional disk arrays can no longer meet the high reliability requirements of large-scale distributed clusters. Data fault tolerance has become an important problem in DSs. In order to ensure the reliability of data processing in DSs and continue to provide users with high-quality services, the performance optimization of fault-tolerant mechanisms has become an important research content in DSs. Aiming at the distributed core storage framework proposed in this paper, the fault-tolerant mechanism of its client module and metadata storage management module is designed and implemented, and the performance of the DS is tested through the performance optimization experiment of fault-tolerant mechanism. The experimental results of Prime-based and neural network-based data inpainting show that the neural network-based data inpainting technology can reduce the network cost of data inpainting. The experimental results of distributed computing delay based on radial basis neural network algorithm, C.T. algorithm and Pedone algorithm show that the delay time of RBF algorithm tends to converge with the increase of MRR.

show abstract

DLFT: Data and Layout Aware Fault Tolerance Framework for Big Data Transfer Systems

Cited by 5 publications

References 16 publications

Gorilla Troops Optimizer Based Fault Tolerant Aware Scheduling Scheme for Cloud Environment

Gorilla Troops Optimizer Based Fault Tolerant Aware Scheduling Scheme for Cloud Environment

CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer Systems

Performance Optimization Technology of Fault Tolerance Mechanism in Distributed System Based on Neural Network

Contact Info

Product

Resources

About