Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication

Abstract

Abstract-The elasticity of Cloud infrastructures makes them a suitable platform for execution of deadline-constrained workflow applications, because resources available to the application can be dynamically increased to enable application speedup. Existing research in execution of scientific workflows in Clouds either try to minimize the workflow execution time ignoring deadlines and budgets or focus on the minimization of cost while trying to meet the application deadline. However, they implement limited contingency strategies to correct delays caused by underestimation of tasks execution time or fluctuations in the delivered performance of leased public Cloud resources. To mitigate effects of performance variation of resources on soft deadlines of workflow applications, we propose an algorithm that uses idle time of provisioned resources and budget surplus to replicate tasks. Simulation experiments with four well-known scientific workflows show that the proposed algorithm increases the likelihood of deadlines being met and reduces the total execution time of applications as the budget available for replication increases.

Adaptive time-based coordinated checkpointing for cloud computing workfl ows

“…The CloudSim is currently the most sophisticated discrete event simulator for Clouds; it is an open source, scalable and low simulation overhead simulator [11]. It is used in many papers to simulate the scientific workflows [31], [32], [33].…”

Scientific Workflow Scheduling with Provenance Data in a Multisite Cloud

“…In addition, based on the measured scheduling time, we estimate that when the number of tasks is less than one million, the scheduling time of DIM with the improved implementation is less than 10% of the total execution time, which is acceptable. According to [10], an SWf of a million tasks is already much bigger than a very large SWf (1000 tasks). Although the scheduling time of DIM may be higher than those of MCT and OLB, the execution time of SWfs corresponds to DIM is much smaller than those of MCT and OLB as explained in the experiments.…”

Budget-Aware Scheduling Algorithms for Scientific Workflows with Stochastic Task Weights on Heterogeneous IaaS Cloud Platforms

“…[2] provides guidelines and analysis to understand cost optimization in scientific workflow scheduling by surveying existing approaches in Cloud computing. Although the multi-objective offline scheduling problem that consists in meeting deadlines and respecting a budget has been extensively studied for deterministic workflows ( [7], [15], [1], for example), it has received much less attention in a stochastic context. In a provider-centered point of view, [16] proposed a framework with the objective of meeting deadlines minimizing the impact of the execution of the workflow's execution on the cluster.…”