Scheduling large-scale scientific workflow on virtual machines with different numbers of vCPUs

Wu, Hao; Chen, Xin; Song, Xiaoyu; Zhang, Chi; Guo, He

doi:10.1007/s11227-020-03273-3

Cited by 8 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Authors in References 40‐46 used makespan and cost minimization for workflow scheduling with deadline and budget constraint; authors have not simulated in cloud‐based simulator.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A new meta‐heuristic approach for load aware‐cost effective workflow scheduling

Ram

Srivastava

Mishra

2022

Concurrency and Computation

View full text Add to dashboard Cite

Workflow scheduling is an important way to manage the execution of a workflow. It introduces the concept of providing suitable resources to workflow tasks in order to finish workflow execution and meet the user's objectives. However, the problem becomes more complex when scheduling must balance two conflicting objectives, such as minimizing execution cost and maximizing load across all computing resources. A workflow has many interdependent tasks, and the cloud datacenter has many computing resources to execute the workflow. There can be an asymptotically infinite number of mappings of tasks-to-computing resources. Every mapping produces different execution costs with different workloads on computing resources. The main challenge for the researcher is to develop an intelligent scheduling algorithm to identify an optimal mapping that produces minimal execution cost with fair workload distribution on resources. We developed a novel meta-heuristic algorithm named Investment-Based Optimization (IBO) to identify an optimal mapping. The IBO algorithm was first tested on optimization benchmark functions and then simulated in CloudSim to see its performance for scheduling workflows. Finally, IBO was tested over Montage, Epigenomics, Sipht, and a sample workflow, and it was found that IBO reduces execution costs by 33%, 16%, 16.36%, and 20% with a fair workload distribution.

show abstract

“…Authors in References 40‐46 used makespan and cost minimization for workflow scheduling with deadline and budget constraint; authors have not simulated in cloud‐based simulator.…”

Section: Related Workmentioning

confidence: 99%

“…Wu et al 43 created a novel scheduling algorithm with DAG splitting approach and used for cost minimization of the workflow application. Author attempted to bring the balance between topology of the DAG‐workflow and parallelism.…”

Section: Related Workmentioning

confidence: 99%

A new meta‐heuristic approach for load aware‐cost effective workflow scheduling

Ram

Srivastava

Mishra

2022

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…As a matter of fact, existing engines are designed to run one workflow of one end-user, and therefore make local optimizations like for example makespan minimization [1], [5], or cost optimization [3], [4], [19], [20], without any knowledge of the physical infrastructure, while assuming that an infinite number of resources are available. Yet, when many users share the same infrastructure, applying a set of local optimizations, at the level of one workflow, leads to a suboptimal utilization of the infrastructure in many dimensions (e.g., fairness, energy efficiency, etc.).…”

Section: Heterogeneitymentioning

confidence: 99%

Handling heterogeneous workflows in the Cloud while enhancing optimizations and performance

Cadorel

Coullon

Menaud

2022

2022 IEEE 15th International Conference on Cloud Computing (CLOUD)

View full text Add to dashboard Cite

The goal of a workflow engine is to facilitate the writing, the deploying, and the execution of a scientific workflow (i.e., graph of coarse-grain and heterogeneous tasks) on distributed infrastructures. With the democratization of the Cloud paradigm, many workflow engines of the state of the art offer a way to execute workflows on distant data centers by using the Infrastructure-as-a-Service (IaaS) or the Function-asa-Service (FaaS) services of Cloud providers. Hence, workflow engines can take advantage of the (presumably) infinite resources and the economical model of the Cloud. However, two important limitations lie in this vision of Cloud-oriented workflow engines. First, by using existing services of Cloud providers, and by managing the workflows at the user side, the Cloud providers are unaware of both the workflows and their user needs, and cannot apply specific resource optimizations to their infrastructure. Second, for the same reasons, handling the heterogeneity of tasks (different operating systems) in workflows necessarily degrades either the transparency for the users (who must provision different types of resources), or the completion time performance of the workflows, because of the stacking of virtualization layers. In this paper, we tackle these two limitations by presenting a new Cloud service dedicated to scientific workflows. Unlike existing workflow engines, this service is deployed and managed by the Cloud providers, and enables specific resource optimizations and offers a better control of the heterogeneity of the workflows. We evaluate our new service in comparison to Argo, a well-known workflow engine of the literature based on FaaS services. This evaluation was made on a real distributed experimental platform with a realistic and complex scenario.

show abstract

“…Makespan and Cost minimization of the workflow execution have been considered by many authors in References 54‐62 without any budget or deadline constraints. Authors in References 63‐69, considered the minimization of makespan and cost under the budget and deadline constraint; authors have not considered the simulator of cloud computing environment. Authors in References 70‐72 focused on minimization of execution cost, completion time and energy.…”

Section: Application Of Gtlbo In Workflow Scheduling Problem On Cloudmentioning

confidence: 99%

“…Zhang et al 71 created a new variant of PSO called SPSO and simulated for completion time and energy optimization. Wu et al 66 developed a new scheduling approach with DAG splitting method for cost minimization. Author tried to maintain the balance between parallelism and the topology of the DAG‐based workflow.…”

Section: Application Of Gtlbo In Workflow Scheduling Problem On Cloudmentioning

confidence: 99%

A variant of teaching‐learning‐based optimization and its application for minimizing the cost of Workflow Execution in the Cloud Computing

Ram

Srivastava

Mishra

2021

Concurrency and Computation

View full text Add to dashboard Cite

Teaching learning‐based optimization (TLBO) was proposed by Rao to solve optimization problems. It is based on the theory of teaching‐learning mechanism. Although it performs well in unimodal problems yet its performance is not good in multimodal problems. To further improve this algorithm's performance and make it suitable for both unimodal problems and multimodal problems, we made some major changes in the theory and the algorithm's operators. The proposed algorithm is able to capture diverse optimal solutions in less number of iterations and is very good for solving multimodal problems. This newly created variant of TLBO is named generalized TLBO (GTLBO). The performance of GTLBO is tested on CEC−06, 2019 benchmark functions and other 15 classical benchmark functions, and it is found that the proposed algorithm is performing better comparatively. Then it is simulated for solving the workflow scheduling problem in CloudSim. Standard scientific workflow applications as Montage, Epigenomics, Sipht, and a sample workflow are used as dataset to test algorithms' performance in cloud environments. Our proposed approach, GTLBO, provides the proper distribution of workloads and offers minimal execution‐cost for the workflow applications. Results reflect the supremacy of the proposed algorithm GTLBO comparatively.

show abstract

Scheduling large-scale scientific workflow on virtual machines with different numbers of vCPUs

Cited by 8 publications

References 28 publications

A new meta‐heuristic approach for load aware‐cost effective workflow scheduling

A new meta‐heuristic approach for load aware‐cost effective workflow scheduling

Handling heterogeneous workflows in the Cloud while enhancing optimizations and performance

A variant of teaching‐learning‐based optimization and its application for minimizing the cost of Workflow Execution in the Cloud Computing

Contact Info

Product

Resources

About