Scheduling Multiple Parameter Sweep Workflow Instances on the Grid

Smanchat, Sucha; Indrawan, Maria; Ling, Sea; Enticott, Colin; Abramson, David

doi:10.1109/e-science.2009.49

Cited by 19 publications

(15 citation statements)

References 19 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When combined with the Nimrod/K engine, this will provide a very powerful distributed computing platform. Second, we are designing and building arrange of scheduling heuristics that work with the data transport and parallel computing models discussed [10] [27]. Third, we are building interfaces with powerful display technologies, and these allow a user to be immersed in the optimization process.…”

Section: Discussionmentioning

confidence: 99%

Robust workflows for science and engineering

Abramson

Bethwaite

Enticott

et al. 2009

Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Robust workflows for science and engineering

Abramson

Bethwaite

Enticott

et al. 2009

Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, the HEFT, min-min and max-min algorithms degrade to the MCT algorithm for this kind of tasks. Some other solutions [35,37,39] for SWf scheduling also focus on single site execution. These techniques do not consider the time to generate provenance data.…”

Section: Related Workmentioning

confidence: 99%

Scientific Workflow Scheduling with Provenance Data in a Multisite Cloud

Liu

Pacitti

Valduriez

et al. 2017

Transactions on Large-Scale Data- And Knowledge-Centered Systems XXXIII

View full text Add to dashboard Cite

Abstract. Recently, some Scientific Workflow Management Systems (SWfMSs) with provenance support (e.g. Chiron) have been deployed in the cloud. However, they typically use a single cloud site. In this paper, we consider a multisite cloud, where the data and computing resources are distributed at different sites (possibly in different regions). Based on a multisite architecture of SWfMS, i.e. multisite Chiron, and its provenance model, we propose a multisite task scheduling algorithm that considers the time to generate provenance data. We performed an extensive experimental evaluation of our algorithm using Microsoft Azure multisite cloud and two real-life scientific workflows (Buzz and Montage). The results show that our scheduling algorithm is up to 49.6% better than baseline algorithms in terms of total execution time.

show abstract

“…Our previous work in [8,13] addressed these issues but did not consider the data transfer time. In this paper, we explain our improved algorithm and describe a conceptual approach to implement a scheduling algorithm for a scheduler.…”

Section: Introductionmentioning

confidence: 96%

“…Since multiple instances of a parameter sweep workflow can be executed in parallel [7], the scheduling algorithm needs to be aware of the resource contention caused by a large number of tasks sharing the same set of resources. In this paper, we present an improved version of our algorithm proposed in [8] to achieve better performance of parameter sweep workflow execution. We also propose an implementation of a scheduler and its interaction with the scheduling algorithm, and highlight the issues encountered during our scheduler development.…”

Section: Introductionmentioning

confidence: 98%

A Scheduler based on Resource Competition for Parameter Sweep Workflow

Smanchat

Indrawan

Ling

et al. 2011

Procedia Computer Science

Self Cite

View full text Add to dashboard Cite

Grid workflow scheduling has been a prevalent field of research in order to allocate scientific workflow tasks to grid resources. To actuate these grid workflow scheduling algorithms, schedulers need to be developed for grid workflow management systems. A scheduler is a component that gathers information, such as estimated execution times and lists of available grid resources, as inputs for scheduling algorithms. Once a grid schedule is generated, the scheduler uses it to allocate grid resources to the tasks in the workflow. This is even more complicated for parameter sweep workflow scheduling. As parameter sweep workflows are repeatedly executed a number of times with different inputs, to schedule them in parallel, the scheduler must be able to handle multiple workflow instances and multiple scheduling iterations. In this paper, we present a scheduling algorithm for parameter sweep workflows and suggest an implementation of a scheduler for parameter sweep workflows based on the algorithms. We highlight the implementation issues encountered in our experience of scheduler development.

show abstract

Scheduling Multiple Parameter Sweep Workflow Instances on the Grid

Abstract: Abstract

Cited by 19 publications

References 19 publications

Robust workflows for science and engineering

Robust workflows for science and engineering

Scientific Workflow Scheduling with Provenance Data in a Multisite Cloud

A Scheduler based on Resource Competition for Parameter Sweep Workflow

Contact Info

Product

Resources

About