Task Duplication-Based Scheduling Algorithm for Budget-Constrained Workflows in Cloud Computing

Abstract

Workflow scheduling is crucial to the efficient operation of cloud platforms, and has attracted a lot of attention. Up to now, many algorithms have been reported to schedule workflows with budget constraints, so as to optimize workflows' makespan on cloud resources. Nevertheless, the hourly-based billing model in cloud computing is an ongoing challenge for workflow scheduling that easily results in higher makespan or even infeasible solutions. Besides, due to data constraints among workflow tasks, there must be a lot of idle slots in cloud resources. Few works adequately exploit these idle slots to duplicate tasks' predecessors to shorten their completion time, thereby minimizing workflow's makespan while ensuring its budget constraint. Motivated by these, we propose a task duplication based scheduling algorithm, namely TDSA, to optimize makespan for budget-constrained workflows in cloud platforms. In TDSA, two novel mechanisms are devised: 1) a dynamic sub-budget allocation mechanism, it is responsible for recovering unused budget of scheduled workflow tasks and redistributing remaining budget, which is conducive to using more expensive/powerful cloud resources to accelerate completion time of unscheduled tasks; and 2) a duplication-based task scheduling mechanism, which strives to exploit idle slots on resources to selectively duplicate tasks' predecessors, such advancing these tasks' completion time while trying to ensuring their sub-budget constraints. At last, we carry out four groups of experiments, three groups on randomly generated workflows and another one on actual workflows, to compare the proposed TDSA with four baseline algorithms. Experimental results confirm that the TDSA has an overwhelming superiority in advancing the workflows' makespan and improving the utilization of cloud computing resources. INDEX TERMS Cloud computing, task duplication, workflow scheduling, resource provision, heuristic mechanism.

Epigenomics Scientific Big Data Workflow Scheduling for Cancer Diagnosis in Health Care Using Heterogeneous Computing Environment

“…Furthermore, there are many scheduling approaches are available for scheduling workflow applications in which the heuristic-based approach generates results in polynomial time with acceptable performance [9]. Heuristicbased scheduling approaches are further divided as list [10][11][12][13][14][15][16][17], duplication [18][19][20], and clustering based scheduling algorithms [21][22][23]. This section discusses the strengths and weaknesses of different heuristicbased workflow scheduling algorithms in a heterogeneous computing environment.…”

“…The scheduler uses the clustering method to reduce communication costs by grouping together communication-intensive dependent tasks and assigning them to the same VM/processor [18][19][20]. The scheduler improves the parallelism degree in the duplication strategy by assigning a key subtask on many processors [21][22][23].…”

Optimized intellectual resource scheduling using deep reinforcement Q‐learning in cloud computing

“…Even though this technique improves the response time it increases resource utilization by 12%. Yao et al 26 provided a task duplication‐based scheduling (TDBS) algorithm to optimize the cost‐related workflows in the cloud platform. The unused budget is recovered using a dynamic sub‐budget allocation strategy and the idle resource slots are reused via a duplication‐based TS mechanism.…”

“…Several different algorithms were introduced for minimizing the energy cost by using task scheduling (TS) and/or resource provisioning (RP) 13‐25 . But scalability issues were experienced or TS with task dependencies were not taken into consideration, which is a vital aspect ensuring precise parallel execution of tasks in parallel 26,27 . Moreover, offline algorithms were not able to deal with larger inputs and accomplish the changes as patterns of user requests have large variations.…”

“…[13][14][15][16][17][18][19][20][21][22][23][24][25] But scalability issues were experienced or TS with task dependencies were not taken into consideration, which is a vital aspect ensuring precise parallel execution of tasks in parallel. 26,27 Moreover, offline algorithms were not able to deal with larger inputs and accomplish the changes as patterns of user requests have large variations.…”