Hierarchical Master-Worker Skeletons

Abstract

Master-worker systems are a well-known and often applicable scheme for the parallel evaluation of a pool of tasks, a work pool. The system consists of a master process managing a set of worker processes. After an initial phase with a fixed amount of tasks for each worker, further tasks are distributed in reply to results sent back by the workers. As this setup quickly leads to a bottleneck in the master process, the paper investigates techniques for hierarchically nesting the basic master-worker scheme. We present implementations of hierarchical master-worker skeletons, and how to automatically calculate parameters of the nested skeleton for good performance.Nesting master-worker systems is nontrivial especially in cases where new tasks are dynamically created from previous results (typically breadthor depth-first tree search algorithms). We discuss how to handle dynamically growing pools in a hierarchy and present a declarative implementation for nested master-worker systems with dynamic task creation.The skeletons are experimentally evaluated with two typical test programs. We analyse their runtime behaviour and the effects of different hierarchies on runtimes via trace visualisations.

Eden – Parallel Functional Programming with Haskell

“…It splits the input list into two halves, keeps the first half for local evaluation and creates a child process on PE 2 for sorting the second half. The remaining ticket list [3..noPe] is unshuffled into the two lists [3,5,7] and [4,6,8]. The first sublist is kept locally while the child process gets the second one.…”

“…For the example tree in Figure 19 we observe that successors 3 2 = [7,8,9] and path 3 9 = [1,2]. If we neglect the case that the problem size might not be large enough to supply each process with work, the offline divide-and-conquer skeleton can be defined as follows:…”

“…We will here exemplarily show how a hierarchical master worker skeleton can elegantly be defined in Eden. For details see [7,50]. As a matter of principle, a nested master-worker system can be defined by using the simple master worker skeleton defined above as the worker function for the upper levels.…”

“…An Eden implementation of the large-scale map-and-reduce programming model proposed by Google [18] has been investigated in [6,4]. Hierarchical master-worker schemes with several layers of masters and submasters have been presented in [7]. A sophisticated distributed workpool has been presented in [19].…”

A Skeleton for Distributed Work Pools in Eden

“…Data transformation/reduction and exhaustive tree search (not discussed here) can also be implemented using a hierarchical master-worker systems (as shown and analysed in [BDLP08]). Our distributed work pool skeleton is tailored 5 to the more interesting case of tree search problems which look for an optimal solution.…”