Fault-Tolerant Scheduling for Scientific Workflows in Cloud Environments

Abstract

Abstract-Executing clustered tasks has proven to be an efficient method to improve the computation of Scientific Workflows (SWf) on clouds. However, clustered tasks has a higher probability of suffering from failures than a single task. Therefore, fault tolerance in cloud computing is extremely essential while running large-scale scientific applications. In this paper, a new heuristic called Cluster based Heterogeneous Earliest Finish Time (C-HEFT) algorithm to enhance the scheduling and fault tolerance mechanism for SWf in highly distributed cloud environments is proposed. To mitigate the failure of clustered tasks, this algorithm uses idle-time of the provisioned resources to resubmit failed clustered tasks for successful execution of SWf. Experimental results show that the proposed algorithm have convincing impact on the SWf executions and also drastically reduce the resource waste compared to existing task replication techniques. A trace based simulation of five real SWf shows that this algorithm is able to sustain unexpected task failures with minimal cost and makespan.

An Automated Pipeline for Advanced Fault Tolerance in Edge Computing Infrastructures

“…Unfortunately, this process requires an excess of resources and thus the overall cost is significantly increased. Resubmission [17] refers to the process of executing a failed task from the start. This process may be repeated on the same computational node or another one that has been chosen.…”

RRFT: A Rank-Based Resource Aware Fault Tolerant Strategy for Cloud Platforms

“…Though the proposed elastic resource provisioning mechanism based on primary-backup model improves the resource utilization in the context of fault tolerant, the faults that occur at the physical server level cannot be handled by the proposed scheduling algorithms. Authors in [30] have proposed similar fault tolerant scientific work-flow scheduling algorithm considering the spot and on-demand instances on the cloud.…”

Real-time and dynamic fault-tolerant scheduling for scientific workflows in clouds

“…Resubmission and replication are two fundamental FT mechanisms that have been effectively applied in the cluster [13,14], grid [15][16][17], and cloud [6,18,19]. Resubmission resubmits a task after a failure happens, which can effectively reduce resource consumption.…”

“…Resubmission resubmits a task after a failure happens, which can effectively reduce resource consumption. However, it introduces a longer task execution time, resulting in hardly satisfying the deadline constraint [19]. Alternatively, replication duplicates multiple task copies and allocates them to different computational units [18].…”