Hierarchical approach to optimization of parallel matrix multiplication on large-scale platforms

Hasanov, Khalid; Quintin, Jean-Noël; Lastovetsky, Alexey

doi:10.1007/s11227-014-1133-x

Cited by 18 publications

(11 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gahvari and Gropp [73] apply performance models to study two exascale use-cases (FFT and multigrids) and conclude that scalability of these applications is limited by interconnect bandwidth. Hasanov et al [88] present a hierarchical approach to distributed matrix multiplication. Evaluation on existing systems (Grid500 and BlueGene/P) and prediction for future exascale systems show better performance than the state-of-the-art.…”

Section: Scalablementioning

confidence: 99%

The Landscape of Exascale Research

et al. 2020

View full text Add to dashboard Cite

The next generation of supercomputers will break the exascale barrier. Soon we will have systems capable of at least one quintillion (billion billion) floating-point operations per second (10 18 FLOPS). Tremendous amounts of work have been invested into identifying and overcoming the challenges of the exascale era. In this work, we present an overview of these efforts and provide insight into the important trends, developments, and exciting research opportunities in exascale computing. We use a three-stage approach in which we (1) discuss various exascale landmark studies, (2) use data-driven techniques to analyze the large collection of related literature, and (3) discuss eight research areas in depth based on influential articles. Overall, we observe that great advancements have been made in tackling the two primary exascale challenges: energy efficiency and fault tolerance. However, as we look forward, we still foresee two major concerns: the lack of suitable programming tools and the growing gap between processor performance and data bandwidth (i.e., memory, storage, networks). Although we will certainly reach exascale soon, without additional research, these issues could potentially limit the applicability of exascale computing.

show abstract

Section: Scalablementioning

confidence: 99%

The Landscape of Exascale Research

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Hierarchical optimization has been recently applied in the development of linear algebra routines for clusters or distributed systems. Hasanov et al applies a hierarchical approach to improve a Hierarchical SUMMA on large‐scale distributed platforms with components at different speeds. In our case, heterogeneity is considered between the nodes in the system and also within each node (CPU + coprocessors).…”

Section: Work In Processmentioning

confidence: 99%

Guided installation of basic linear algebra routines in a cluster with manycore components

Cuenca

García

Giménez

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary Computational systems are nowadays composed of basic computational components that share multiprocessors and coprocessors of different types, typically several graphics processing units (GPUs) or many integrated cores (MICs), and those computational components are combined in heterogeneous clusters of nodes with different characteristics, including coprocessors of different types, with varying numbers of nodes at different speeds. The software previously developed and optimized for simpler system needs to be redesigned and reoptimized for these new, more complex systems. The adaptation to hybrid multicore + multiGPU and multicore + multiMIC of autotuning techniques for basic linear algebra routines is analyzed. The matrix‐matrix multiplication kernel, which is optimized for different computational system components through guided experimentation, is studied. The routine is installed for each node in the cluster, and the information generated from individual installations may be used for a hierarchical installation in a cluster. The basic matrix‐matrix multiplication may, in turn, be used inside higher level routines, which delegate their efficient execution to the optimization of the lower level routine. Experimental results are satisfactory in different multicore + multiGPU and multicore + multiMIC systems. So the guided search of execution configurations for satisfactory execution times proves to be a useful tool for heterogeneous systems, where the complexity of the system means a correct use of highly efficient routines and libraries is difficult.

show abstract

“…This optimization technique is inspired by our previous study on the optimization of the communication cost of parallel matrix multiplication on largescale distributed memory platforms [16].…”

Section: Hierarchical Optimization Of Mpi Broadcast Algorithmsmentioning

confidence: 99%

“…In contrast to this approach, we focus on the scale of HPC platforms rather than their specific complexity and propose a solution that is universally applicable to all HPC platforms. The proposed solution is inspired by our previous study on parallel matrix multiplication on large-scale distributed memory platforms [16]. It provides a simple and general technique to optimize the legacy scientific MPI-based applications without redesigning them.…”

Section: Introductionmentioning

confidence: 99%