Matrix multiplication on heterogeneous platforms

Beaumont, Olivier; Boudet, Vincent; Rastello, Fabrice; Robert, Yves

doi:10.1109/71.963416

Cited by 127 publications

(119 citation statements)

References 21 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…They report several numerical simulations. As pointed out in the introduction, theoretical results for matrix multiplication and LU decomposition on 2D-grids of heterogeneous processors are reported in [5], while extensions to general 2D partitioning are considered in [6]. See also Lastovetsky and Reddy [31] for another partitioning approach.…”

Section: Inriamentioning

confidence: 99%

“…With this hypothesis, minimizing the total communication cost amounts to minimizing the total communication volume. Unfortunately, this problem has been shown NP-complete as well [6]. Note that even under the optimistic assumption that all communications at a given step of the algorithm can take place in parallel, the problem remains NP-complete [7].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

This paper is aimed at designing efficient parallel matrix-product algorithms for heterogeneous master-worker platforms. While matrix-product is well-understood for homogeneous 2D-arrays of processors (e.g., Cannon algorithm and ScaLAPACK outer product algorithm), there are three key hypotheses that render our work original and innovative: -Centralized data. We assume that all matrix files originate from, and must be returned to, the master. The master distributes both data and computations to the workers (while in ScaLAPACK, input and output matrices are initially distributed among participating resources). Typically, our approach is useful in the context of speeding up MATLAB or SCILAB clients running on a server (which acts as the master and initial repository of files).-Heterogeneous star-shaped platforms. We target fully heterogeneous platforms, where computational resources have different computing powers. Also, the workers are connected to the master by links of different capacities. This framework is realistic when deploying the application from the server, which is responsible for enrolling authorized resources.-Limited memory. Because we investigate the parallelization of large problems, we cannot assume that full matrix panels can be stored in the worker memories and re-used for subsequent updates (as in ScaLAPACK). The amount of memory available in each worker is expressed as a given number m i of buffers, where a buffer can store a square block of matrix elements. The size q of these square blocks is chosen so as to harness the power of Level 3 BLAS routines: q = 80 or 100 on most platforms.We have devised efficient algorithms for resource selection (deciding which workers to enroll) and communication ordering (both for input and result messages), and we report a set of numerical experiments on various platforms atÉcole Normale Supérieure de Lyon and Résumé : Ce papier a pour objectif la définition d'algorithmes efficaces pour le produit de matrices en parallèle sur plate-formes maître-esclaves hétérogènes. Bien que le produit de matrices soit bien compris pour des grilles bi-dimensionnelles de processeurs homogènes (cf. l'algorithme de Cannon et le produit externe de ScaLAPACK), trois hypothèses rendent notre travail original: -Données centralisées. Nous supposons que toutes les matrices résident originellement sur le maître, et doivent yêtre renvoyées. Le maître distribue données et calculs aux esclaves (alors que dans ScaLAPACK, les matrices initiales et résultats sont initiallement distribuées aux processeurs participant). Typiquement, notre approche est justifiée dans le contexte de l'accélération de clients MATLAB ou SCILAB s'exécutant sur un serveur (qui se comporte comme le maître et détient initiallement les données).-Plates-formes hétérogènes enétoile. Nous nous intéressonsà des plates-formes complètement hétérogènes dont les ressources de calculs ont des puissances de calcul différentes et dont les esclaves sont reliés au maître par des liens de capacités différentes. Ce cadre d...

show abstract

Section: Inriamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Revisiting Matrix Product on Master-Worker Platforms

Dongarra

Pineau

Robert

et al. 2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…This heterogeneity is seldom planned, arising mainly as a result of technology evolution over time and computer market sales and trends. Commodity off-the-shelf heterogeneous clusters of computers can realize a very high level of aggregate performance [6], and it is expected that these clusters will represent a tool of choice for the scientific community devoted to high-dimensional image analysis in remote sensing and other fields [7][8][9]. It is also worth noting that significant opportunities to exploit heterogeneous computing techniques are still available in the analysis of high-dimensional image data sets [10].…”

Section: Introductionmentioning

confidence: 99%

Parallel morphological processing of hyperspectral image data on heterogeneous networks of computers

Plaza

2006

Proceedings 20th IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…It is usually a difficult design task to come up with a practical and efficient heterogeneous counterpart of a homogeneous parallel algorithm on HNOCs. The problem of optimal heterogeneous data distribution has proved to be NP-complete even for such a simple linear algebra kernel as matrix multiplication on HNOCs [6]. Once the heterogeneous parallel algorithm is designed, its portable and efficient implementation on heterogeneous platforms requires writing of a lot of complex code to automate several tedious and error-prone tasks [7].…”

Section: Introductionmentioning

confidence: 99%

HeteroMPI+ScaLAPACK: Towards a ScaLAPACK (Dense Linear Solvers) on Heterogeneous Networks of Computers

Reddy¹,

Lastovetsky

2006

High Performance Computing - HiPC 2006

View full text Add to dashboard Cite

Abstract. The paper presents a tool that ports ScaLAPACK programs designed to run on massively parallel processors to Heterogeneous Networks of Computers. The tool converts ScaLAPACK programs to HeteroMPI programs. The resulting HeteroMPI programs do not aim to extract the maximum performance from a Heterogeneous Networks of Computers but provide an easy and simple way to execute the ScaLAPACK programs on such networks with good performance improvements. We demonstrate the efficiency of the resulting HeteroMPI programs by performing experiments with a matrix multiplication application on a local network of heterogeneous computers.

show abstract

Matrix multiplication on heterogeneous platforms

Abstract: International audienceno abstrac

Cited by 127 publications

References 21 publications

Revisiting Matrix Product on Master-Worker Platforms

Revisiting Matrix Product on Master-Worker Platforms

Parallel morphological processing of hyperspectral image data on heterogeneous networks of computers

HeteroMPI+ScaLAPACK: Towards a ScaLAPACK (Dense Linear Solvers) on Heterogeneous Networks of Computers

Contact Info

Product

Resources

About