Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization

Malawski, Maciej; Figiela, Kamil; Bubak, Marian; Deelman, Ewa; Nabrzyski, Jarek

doi:10.1155/2015/680271

Cited by 46 publications

(22 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We consider this to be a global optimization problem that is different from non-trivial challenge of selecting appropriate storage systems and scheduling workflows within a homogeneous cloud. In the future we aim to combine these local and global approaches, using the methods presented in this paper and in our related research [38].…”

Section: Discussionmentioning

confidence: 99%

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

Bryk¹,

Malawski

Juve³

et al. 2015

J Grid Computing

Self Cite

View full text Add to dashboard Cite

This paper focuses on data-intensive workflows and addresses the problem of scheduling workflow ensembles under cost and deadline constraints in Infrastructure as a Service (IaaS) clouds. Previous research in this area ignores file transfers between workflow tasks, which, as we show, often have a large impact on workflow ensemble execution. In this paper we propose and implement a simulation model for handling file transfers between tasks, featuring the ability to dynamically calculate bandwidth and supporting a configurable number of replicas, thus allowing us to simulate various levels of congestion. The resulting model is capable of representing a wide range of storage systems available on clouds: from inmemory caches (such as memcached), to distributed file systems (such as NFS servers) and cloud storage (such as Amazon S3 or Google Cloud Storage). We observe that file transfers may have a significant impact on ensemble execution; for some applications up to 90 % of the execution time is spent on file transfers. Next, we propose and evaluate a novel scheduling algorithm that minimizes the number of transfers by taking advantage of data caching and file locality. We find that for data-intensive applications it performs better than other scheduling algorithms. Additionally, we modify the original scheduling algorithms to effectively operate in environments where file transfers take non-zero time.

show abstract

Section: Discussionmentioning

confidence: 99%

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

Bryk¹,

Malawski

Juve³

et al. 2015

J Grid Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Malawski et al present a mathematical model that optimizes the cost of scheduling workflows under a deadline constraint. It considers a multicloud environment where each provider offers a limited number of heterogeneous VMs, and a global storage service is used to share intermediate data files.…”

Section: Surveymentioning

confidence: 99%

A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments

Rodríguez

Buyya

2016

Concurrency and Computation

196

119

View full text Add to dashboard Cite

Summary Large‐scale scientific problems are often modeled as workflows. The ever‐growing data and compute requirements of these applications has led to extensive research on how to efficiently schedule and deploy them in distributed environments. The emergence of the latest distributed systems paradigm, cloud computing, brings with it tremendous opportunities to run scientific workflows at low costs without the need of owning any infrastructure. It provides a virtually infinite pool of resources that can be acquired, configured, and used as needed and are charged on a pay‐per‐use basis. However, along with these benefits come numerous challenges that need to be addressed to generate efficient schedules. This work identifies these challenges and studies existing algorithms from the perspective of the scheduling models they adopt as well as the resource and application model they consider. A detailed taxonomy that focuses on features particular to clouds is presented, and the surveyed algorithms are classified according to it. In this way, we aim to provide a comprehensive understanding of existing literature and aid researchers by providing an insight into future directions and open issues.

show abstract

“…Malawski et al [34] proposed a cost-optimization model for scheduling scientific workflows in IaaS clouds, using mathematical programming languages (AMPL and CMPL) which optimizes the cost under a deadline constraint in multi-cloud environment, where each provider offers a limited number of heterogeneous VMs, and cloudobjectstoresuchasAmazonS3 to share intermediate data files. Their method proposes different models such as application model, infrastructure model, and the scheduling model as mixed integer programming (MIP).…”

Section: Multilevel Deadline-constrained Scientific Workflowsmentioning

confidence: 99%

Workflow Scheduling Techniques and Algorithms in IaaS Cloud: A Survey

Chakravarthi

Vijayakumar

2018

IJECE

View full text Add to dashboard Cite

In the modern era, workflows are adopted as a powerful and attractive paradigm for expressing/solving a variety of applications like scientific, data intensive computing, and big data applications such as MapReduce and Hadoop. These complex applications are described using high-level representations in workflow methods. With the emerging model of cloud computing technology, scheduling in the cloud becomes the important research topic. Consequently, workflow scheduling problem has been studied extensively over the past few years, from homogeneous clusters, grids to the most recent paradigm, cloud computing. The challenges that need to be addressed lies in task-resource mapping, QoS requirements, resource provisioning, performance fluctuation, failure handling, resource scheduling, and data storage. This work focuses on the complete study of the resource provisioning and scheduling algorithms in cloud environment focusing on Infrastructure as a service (IaaS). We provided a comprehensive understanding of existing scheduling techniques and provided an insight into research challenges that will be a possible future direction to the researchers.

show abstract

Scheduling Multilevel Deadline-Constrained Scientific Workflows on Clouds Based on Cost Optimization

Cited by 46 publications

References 25 publications

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

Storage-aware Algorithms for Scheduling of Workflow Ensembles in Clouds

A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments

Workflow Scheduling Techniques and Algorithms in IaaS Cloud: A Survey

Contact Info

Product

Resources

About