Scheduling Concurrent Bag-of-Tasks Applications on Heterogeneous Platforms

Abstract

Abstract-Scheduling problems are already difficult on traditional parallel machines, and they become extremely challenging on heterogeneous clusters. In this paper, we deal with the problem of scheduling multiple applications, made of collections of independent and identical tasks, on a heterogeneous master-worker platform. The applications are submitted online, which means that there is no a priori (static) knowledge of the workload distribution at the beginning of the execution. The objective is to minimize the maximum stretch, i.e., the maximum ratio between the actual time an application has spent in the system and the time this application would have spent if executed alone. On the theoretical side, we design an optimal algorithm for the offline version of the problem (when all release dates and application characteristics are known beforehand). We also introduce a heuristic for the general case of online applications. On the practical side, we have conducted extensive simulations and MPI experiments, showing that we are able to deal with very large problem instances in a few seconds. Also, the solution that we compute totally outperforms classical heuristics from the literature, thereby fully assessing the usefulness of our approach.

Minimising the Execution of Unknown Bag-of-Task Jobs with Deadlines on the Cloud

“…Beaumont et al aimed to maximise the number of tasks executed concurrently while ensuring fair resource sharing between applications [3]. Benoit et al took another approach to minimise the application runtime while sharing resources betweens applications [4].…”

Self-Healing of Operational Workflow Incidents on Distributed Computing Infrastructures

“…In these domains, most approaches are clairvoyant, meaning that resource, task, error rate and workload characteristics are precisely known [3,20]. Heuristics are designed by Casanova et al [7] for the case where only data transfer costs are known, on an offline problem though.…”

A Fair Decentralized Scheduler for Bag-of-Tasks Applications on Desktop Grids

“…To discourage users tampering with their application to get better performance, we must provide a scheduler that gives a fair share of the available resources to each user. Similar problems have been addressed in computing Grids [7,14]. However, these schedulers are centralized, and assume perfectly updated information on the whole platform.…”

“…In the context of computing Grids, a common model for such applications is the bag-of-tasks: each application is then described as a set of similar tasks, i.e. which have a common data file size and computing demand [7,21].…”

“…Our study relies on previous work which proposes a peer-to-peer architecture to distribute tasks provided with deadlines [9]. We also build upon a previous study on a centralized scheduler for multiple bag-of-tasks applications on a heterogeneous platform [7].…”