A Scheduler based on Resource Competition for Parameter Sweep Workflow

Abstract

Grid workflow scheduling has been a prevalent field of research in order to allocate scientific workflow tasks to grid resources. To actuate these grid workflow scheduling algorithms, schedulers need to be developed for grid workflow management systems. A scheduler is a component that gathers information, such as estimated execution times and lists of available grid resources, as inputs for scheduling algorithms. Once a grid schedule is generated, the scheduler uses it to allocate grid resources to the tasks in the workflow. This is even more complicated for parameter sweep workflow scheduling. As parameter sweep workflows are repeatedly executed a number of times with different inputs, to schedule them in parallel, the scheduler must be able to handle multiple workflow instances and multiple scheduling iterations. In this paper, we present a scheduling algorithm for parameter sweep workflows and suggest an implementation of a scheduler for parameter sweep workflows based on the algorithms. We highlight the implementation issues encountered in our experience of scheduler development.

An Adaptive Grid Workflow Scheduling Based on Bottleneck Detection and Execution Context

“…These techniques use common resource performance metrics to calculate schedules including estimated execution time (EET) of a task, estimated wait time (EWT) on a Grid resource, and data transfer time (ETT) between Grid resources [11]. However, some algorithms, such as the "Besom" algorithm [5,9], require additional information or context to address specific issues. This section reviews some examples of list scheduling algorithms [11] and identifies the types of information they used.…”

“…By parallelizing the execution of multiple instances of a parameter sweep workflow, the overall execution time can be reduced [8]. However, a bottleneck may occur as the tasks in multiple instances of the parameter sweep workflow require the same set of Grid resources [5,9]. Although some algorithms are designed to handle parameter sweep workflows, they do not specifically address the bottleneck issue [4].…”

Resource scheduling of workflow multi-instance migration based on the shuffled leapfrog algorithm

“…Aiming to resolve the grid workflow scheduling problem, Sucha Smanchat et al proposed a scheduling algorithm for a multi-parameter sweep workflow instance based on resource competition (Smanchat, Indrawan & Ling, 2011). Rizos Sakellariou et al considered resource allocation problems to be a single activity instance of the workflow and set the earliest completion time for a certain activity instance as the goal of their resource scheduling method (Sakellariou, Zbao, Tsiakkouri & Dikaiako, 2007).…”