Data Partitioning with a Functional Performance Model of Heterogeneous Processors

Abstract

In this paper, we address the problem of optimal distribution of computational tasks on a network of heterogeneous computers when one or more tasks do not fit into the main memory of the processors and when relative speeds vary with the problem size. We propose a functional performance model of heterogeneous processors that integrates many essential features of a network of heterogeneous computers having a major impact on its performance such as the processor heterogeneity, the heterogeneity of memory structure, and the effects of paging. Under this model, the speed of each processor is represented by a continuous function of the size of the problem whereas traditional models use single numbers to represent the speeds of the processors. We formulate a problem of partitioning of an nelement set over p heterogeneous processors using this model and design an algorithm of the complexity O(p × log 2 n) solving the problem.

Two-Dimensional Matrix Partitioning for Parallel Computing on Heterogeneous Processors Based on Their Functional Performance Models

“…A set of p curves on this plane (as shown in Figure 3 (b)) will represent the absolute speeds of the processors against variable y given parameter x is fixed; • Apply the set partitioning algorithm [4] to this set of p curves to obtain an optimal distribution.…”

“…The problem of distributing independent chunks of computations over a onedimensional arrangement of heterogeneous processors using this FPM has been studied in [4]. It can be formulated as follows: Given n independent chunks of computations, each of equal size (i.e., each requiring the same amount of work), how can we assign these chunks to p (p<n) physical processors P 1 , P 2 , ..., P p with their respective full FPMs represented by speed functions s 1 (x), s 2 (x), ..., s p (x) so that the workload is best balanced?…”

Dynamic Load Balancing of Parallel Computational Iterative Routines on Platforms with Memory Heterogeneity

“…It has been shown in [13] that it is more accurate to represent performance as a function of problem size, which reflects contributions from both processor and memory. In this paper, we propose a new dynamic load balancing algorithm based on partial functional performance models of processors [3].…”

“…Our dynamic load balancing algorithm is based on functional performance models [13], which are application centric and hardware specific. Functional performance models reflect both processor and memory heterogeneity.…”

Accurate and Efficient Estimation of Parameters of Heterogeneous Communication Performance Models

“…This type of parallel application is often used in practice, for example, in processing of a large amount of image data collected from the hyperspectral sensors on airborne/satellite platforms (Plaza et al 2006). Our application multiplies two dense square matrices, C = A × B, and employs a simple heterogeneous parallel algorithm based on one-dimensional matrix partitioning (see, for example, Lastovetsky and Reddy 2007). As shown in Figure 5, the matrices A and C are horizontally sliced such that the number of elements in a slice is proportional to the speed of the processor owning the slice.…”