Array Decompositions for Nonuniform Computational Environments

Kaddoura, Maher; Ranka, Sanjay; Wang, Albert

doi:10.1006/jpdc.1996.0092

Cited by 30 publications

(19 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their idea is to achieve a perfect load-balance as follows: first they take a fixed layout of processors arranged as a collection of processor columns; then the load is evenly balanced within each processor column independently; next the load is balanced between columns; this is the "heterogeneous block cyclic distribution" of [29]. Another approach is proposed by Crandall and Quinn [20], who propose a recursive partitioning algorithm, and by Kaddoura, Ranka and Wang [28], who refine the latter algorithm and provide several variations. They report several numerical simulations.…”

Section: Inriamentioning

confidence: 99%

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

This paper is aimed at designing efficient parallel matrix-product algorithms for heterogeneous master-worker platforms. While matrix-product is well-understood for homogeneous 2D-arrays of processors (e.g., Cannon algorithm and ScaLAPACK outer product algorithm), there are three key hypotheses that render our work original and innovative: -Centralized data. We assume that all matrix files originate from, and must be returned to, the master. The master distributes both data and computations to the workers (while in ScaLAPACK, input and output matrices are initially distributed among participating resources). Typically, our approach is useful in the context of speeding up MATLAB or SCILAB clients running on a server (which acts as the master and initial repository of files).-Heterogeneous star-shaped platforms. We target fully heterogeneous platforms, where computational resources have different computing powers. Also, the workers are connected to the master by links of different capacities. This framework is realistic when deploying the application from the server, which is responsible for enrolling authorized resources.-Limited memory. Because we investigate the parallelization of large problems, we cannot assume that full matrix panels can be stored in the worker memories and re-used for subsequent updates (as in ScaLAPACK). The amount of memory available in each worker is expressed as a given number m i of buffers, where a buffer can store a square block of matrix elements. The size q of these square blocks is chosen so as to harness the power of Level 3 BLAS routines: q = 80 or 100 on most platforms.We have devised efficient algorithms for resource selection (deciding which workers to enroll) and communication ordering (both for input and result messages), and we report a set of numerical experiments on various platforms atÉcole Normale Supérieure de Lyon and Résumé : Ce papier a pour objectif la définition d'algorithmes efficaces pour le produit de matrices en parallèle sur plate-formes maître-esclaves hétérogènes. Bien que le produit de matrices soit bien compris pour des grilles bi-dimensionnelles de processeurs homogènes (cf. l'algorithme de Cannon et le produit externe de ScaLAPACK), trois hypothèses rendent notre travail original: -Données centralisées. Nous supposons que toutes les matrices résident originellement sur le maître, et doivent yêtre renvoyées. Le maître distribue données et calculs aux esclaves (alors que dans ScaLAPACK, les matrices initiales et résultats sont initiallement distribuées aux processeurs participant). Typiquement, notre approche est justifiée dans le contexte de l'accélération de clients MATLAB ou SCILAB s'exécutant sur un serveur (qui se comporte comme le maître et détient initiallement les données).-Plates-formes hétérogènes enétoile. Nous nous intéressonsà des plates-formes complètement hétérogènes dont les ressources de calculs ont des puissances de calcul différentes et dont les esclaves sont reliés au maître par des liens de capacités différentes. Ce cadre d...

show abstract

Section: Inriamentioning

confidence: 99%

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…LU and QR decomposition have been discussed by Barbosa et al [1]. Static partitioning schemes to map a two-dimensional data matrix onto heterogeneous resources have been investigated by Crandall and Quinn [14], Kaddoura et al [24], and Beaumont et al [3]. The main conclusions of these papers are drawn for three kinds of problems:…”

Section: Resultsmentioning

confidence: 99%

Mapping and load-balancing iterative computations

Legrand

Renard

Robert

et al. 2004

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Abstract-This paper is devoted to mapping iterative algorithms onto heterogeneous clusters. The application data is partitioned over the processors, which are arranged along a virtual ring. At each iteration, independent calculations are carried out in parallel, and some communications take place between consecutive processors in the ring. The question is to determine how to slice the application data into chunks, and to assign these chunks to the processors, so that the total execution time is minimized. One major difficulty is to embed a processor ring into a network that typically is not fully connected, so that some communication links have to be shared by several processor pairs. We establish a complexity result that assesses the difficulty of this problem, and we design a practical heuristic that provides efficient mapping, routing, link-sharing, and data distribution schemes.

show abstract

“…Several studies have been conducted on deploying DD methods within heterogeneous environments [4,5,18,19,22]. In most cases, the problem is reduced to the problem of partitioning some mathematical objects, such as matrices, sets or graphs [9].…”

Section: Related Workmentioning

confidence: 99%

“…First, the computational load can be redistributed between masters. This approach makes it possible to use existing load redistribution strategies [4][5][6]18,19,22].…”

Section: Future Workmentioning

confidence: 99%