Optimal sharing of bags of tasks in heterogeneous clusters

Abstract

We prove that "FIFO" worksharing protocols provide asymptotically optimal solutions to a problem related to sharing a bag of identically complex tasks in a heterogeneous network of workstations (HNOW) N . In the HNOW-Exploitation Problem, one seeks to accomplish as much work as possible on N during a prespecified fixed period of L time units. The worksharing protocols we study are crafted within an architectural model that characterizes N via parameters that measure workstations' computational and communicational powers. The protocols are self-scheduling, in that they determine completely both an amount of work to allocate to each of N 's workstations and a schedule for all related interworkstation communications. A protocol observes a FIFO regimen if it has N 's workstations finish their assigned work, and return their results, in the same order in which they are supplied with their workloads. The optimality of FIFO protocols resides in the fact that they accomplish at least as much work as any other protocol during all sufficiently long worksharing episodes. Simulation experiments indicate that the superiority of FIFO protocols is often observed during worksharing episodes of only a few minutes' duration.

Bag-of-Tasks applications scheduling on volunteer desktop grids with adaptive information dissemination

“…The Bag-of-Tasks (BoT) application model [5], i.e., an embarrassingly parallel application composed of a collection of independent and identical tasks, is often considered as a suitable model for desktop grids [6], [7]. Scheduling concurrent BoT applications is challenging because any delay in completion of any one of the constituent tasks of the bag will affect the overall bag turnaround time.…”

To BSP or not to BSP in heterogeneous NOWs

“…The algorithmics of cluster computing is not yet as well developed as the algorithmics of multiprocessor-based computing: there have been rather few rigorously analyzed studies of scheduling computations on NOWs that are heterogeneous (are HNOWs), in the sense that their workstations may differ in power (i.e., processor and memory speeds). In this paper, we continue the study in [1,20] of the HNOW-Exploitation Problem (HEP, for short), wherein one has £ Supported in part by NSF Grant CCR-00-73401. a large "bag" of roughly equivalent tasks, and one seeks to compute as many tasks as possible on a "rented" HNOW AE during a prespecified fixed period of time.…”

“…In this study, we compare the two strategies under the HiHCoHP model of [9], which characterizes the HNOW AE via a suite of parameters that quantify the relative speeds of AE's workstations and the speed of its underlying network. We bound the exact and asymptotic 1 work-output of a near-optimal COMM-C protocol and compare this with the work-output of the optimal COMP-C FIFO protocol of [1,20]. We find that both strategies achieve their respective goals-the COMP-C strategy accomplishes more work per unit time, while the COMM-C strategy significantly decreases communication overhead.…”

“…clusters, via a suite of parameters that character-ize the powers of both the workstations and networks that make up such clusters. In common with [1,20], we employ a modified HiHCoHP model, "flattened" to represent a single HNOW and augmented with a computation-related parameter. Our indebtedness to [9] automatically implies a debt to the homogeneous LogP and LogGP models of [11,2].…”

“…Among the few rigorously analyzed studies of workallocation plus -scheduling in HNOWs, the closest relatives of the current study are [1,20], which develop a model for COMP-C worksharing protocols that solve the HEP and expose the asymptotic optimality of the FIFO Protocol among these protocols. The several papers exemplified by [7,10], and sources cited therein, study the scheduling of "divisible jobs" in single-level tree networks, which can be viewed as abstractions of computation-centrically scheduled HNOWs.…”