Scheduling Task Parallel Applications for Rapid Turnaround on Enterprise Desktop Grids

Abstract

Desktop Grids are popular platforms for high throughput applications, but due to their inherent resource volatility it is difficult to exploit them for applications that require rapid turnaround. Efficient desktop Grid execution of shortlived applications is an attractive proposition and we claim that it is achievable via intelligent resource selection. We propose three general techniques for resource selection: resource prioritization, resource exclusion, and task duplication. We use these techniques to instantiate several scheduling heuristics. We evaluate these heuristics through trace-driven simulations of four representative desktop Grid configurations. We find that ranking desktop resources according to their clock University of Hawai'i, Manoa, Hawai'i rates, without taking into account their availability history, is surprisingly effective in practice. Our main result is that a heuristic that uses the appropriate combination of resource prioritization, resource exclusion, and task replication can achieve performance within a factor of 1.7 of optimal in practice.

Using Replication for Resilience on Exascale Systems

“…In this work we study the used of replication as a mechanism complementary to checkpoint-rollback. Replication has long been used as a faulttolerance mechanism in distributed systems [11] and more recently in the context of volunteer computing [15]. The idea to use replication together with checkpoint-rollback has been studied in the context of grid computing [22].…”

Scheduling Multiple Parameter Sweep Workflow Instances on the Grid

“…However, the proposed algorithm especially the ranking function needs to be refined. In addition, further improvement can be made using resource exclusion as suggested in [22] and utilising the structure of the workflow to refine the resource competition calculation.…”

Fault-Tolerance Techniques for High-Performance Computing

“…For convenience, we provide a few bibliographical notes in the following lines. Replication has long been used as a fault tolerance mechanism in distributed systems [38], and in the context of volunteer computing [51]. Replication has recently received attention in the context of HPC (High Performance Computing) applications [66,72,31,33].…”