An Integrated Approach for Processor Allocation and Scheduling of Mixed-Parallel Applications

Abstract

Computationally complex applications can often be viewed as a collection of coarse-grained data-parallel tasks with precedence constraints. Researchers have shown that combining task and data parallelism (mixed parallelism) can be an effective approach for executing these applications, as compared to pure task or data parallelism. In this paper, we present an approach to determine the appropriate mix of task and data parallelism, i.e., the set of tasks that should be run concurrently and the number of processors to be allocated to each task. An iterative algorithm is proposed that couples processor allocation and scheduling, of mixedparallel applications on compute clusters so as to minimize the parallel completion time (makespan). Our algorithm iteratively reduces the makespan by increasing the degree of data parallelism of tasks on the critical path that have good scalability and a low degree of potential task parallelism. The approach employs a look-ahead technique to escape local minima and uses priority based backfill scheduling to efficiently schedule the parallel tasks onto processors. Evaluation using benchmark task graphs derived from real applications as well as synthetic graphs shows that our algorithm consistently performs better than CPR and CPA, two previously proposed scheduling schemes, as well as pure task and data parallelism.

Scheduling Parallel Task Graphs on (Almost) Homogeneous Multicluster Platforms

“…Several practical PTG scheduling algorithms based on heuristics have been proposed in the literature [6], [8], [9], [10], [11]. Like the guaranteed algorithms discussed earlier, the algorithms in [6], [8], [9], [10] proceed in two phases.…”

“…These last three algorithms all use a list-scheduling-based task mapping phase by which tasks are mapped to processors in order of decreasing "bottom level" (i.e., distance to the PTG's exit task), accounting for data communication and data redistribution costs. The iCASLB one-step algorithm in [11] was shown to lead to better performance than some two-step algorithms, including CPA, while maintaining reasonable complexity. This algorithm performs allocation and mapping simultaneously by iteratively increasing the allocations of tasks on the critical path, with a look-ahead mechanism to avoid being trapped in local minima, and a backfilling approach to improve the schedule.…”

“…From a theoretical standpoint, although the scheduling problem is NP-complete, algorithms with performance guarantees, defined as the maximum ratio between the produced makespan and the optimal makespan, have been developed in [2], [3], [4], [5]. From a more applied standpoint, many nonguaranteed heuristics have been proposed and shown to lead to good average performance in practice [6], [7], [8], [9], [10], [11].…”

Static Mapping with Dynamic Switching of Multiple Data-Parallel Applications on Embedded Many-Core SoCs

“…Then overlap j1, j2 is extracted as shown in Table 1. Since Tile 1 and Tile 5 are overlapped, overlap 1,5 and overlap 5,1 take 1. On the other hand, overlap 1,6 and overlap 6,1 take 0 because Tile 1 and Tile 6 do not share any cores.…”

“…In other words, an application is assigned a single core. Techniques presented in [2]- [5] take into account data parallelism within applications (intra-application parallelism) as well as application-level parallelism (interapplication parallelism). Their methods perform scheduling and mapping simultaneously, aiming at minimization of schedule length or pipeline throughput.…”