Coordinated rescheduling of Bag‐of‐Tasks for executions on multiple resource providers

Abstract

Abstract-Metaschedulers can distribute parts of a Bag-of-Tasks (BoT) application among various resource providers in order to speed up its execution. The expected completion time of the user application is then calculated based on the run time estimates of all applications running and waiting for resources. However, due to inaccurate run time estimates, initial schedules are not those that provide users with the earliest completion time. These estimates increase the time distance between the first and last tasks of a BoT application, which increases average user response time, especially in multi-provider environments. This paper proposes a coordinated rescheduling algorithm to handle inaccurate run time estimates when executing BoT applications in multi-provider environments. The coordinated rescheduling defines which tasks can have start time updated based on the expected completion time of the entire BoT application. We have also evaluated the impact of system-generated run time estimates to schedule BoT applications on multiple providers. We performed experiments using simulations and a real distributed platform, Grid'5000. From our experiments, we obtained reductions of up to 5% and 10% for response time and slowdown metrics respectively by using coordinated rescheduling over a traditional rescheduling solution. Moreover, coordinated rescheduling requires little modification of existing scheduling systems. System-generated predictions, on the other hand, are more complex to be deployed and may not reduce response times as much as when using coordinated rescheduling.

Modeling Many-Task Computing Workloads on a Petaflop IBM Blue Gene/P Supercomputer

“…We claim this is due to the scheduling capability of the framework (e.g. Falkon [22]). When the BOT stays on the margin, it means the framework schedules these tasks with the highest efficiency (tasks are executed immediately after they arrive).…”

“…Task replication could be considered as a particular type of rescheduling, which has the drawbacks of wasting resources and causing consistent problems. Marco et al [22] proposed a coordinated rescheduling algorithm for BOT applications and an evaluation of the impact of run time when scheduling these applications across multiple providers. A. Oprescu et al [23] developed a scheduler, BaTS, to schedule BOTs in dynamic cloud environment under the user-defined budget constraint.…”

From volunteer to trustable computing: Providing QoS-aware scheduling mechanisms for multi-grid computing environments

“…This is evaluated in [57] where a coordinated BoT rescheduling algorithm that deals with the estimated completion time of the entire BoT application is presented.…”

Agent-based hierarchical approach for executing bag-of-tasks in clouds

“…In comparison with the message passing model, they can be highly parallelized and executed in any acceptable order. Such tasks are adopted in multiple scientific research: computational biology, image processing, and massive search engines [28]. The scheduling of independent tasks has been the subject of research over the years [12,16], and it was shown that the scheduling of them on a set of heterogeneous computing resources is a NP-complete problem.…”

“…When tasks are dispatched between multiple clusters, this inaccuracy grows, since rescheduling is performed independently by each resource provider. This time difference is called stretch factor [28]. The inaccuracy of predictions can be reduced by such methods as analyzing the scheduling traces (application independent, but problems with heterogeneous workloads) or application profiling (generating run-time predictions, but requires source-code access).…”