HeteroMPI+ScaLAPACK: Towards a ScaLAPACK (Dense Linear Solvers) on Heterogeneous Networks of Computers

Reddy, Ravi; Lastovetsky, Alexey

doi:10.1007/11945918_27

Cited by 8 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The HeteroScaLAPACK project 7 aims at developing a parallel dense linear algebra package for heterogeneous architectures on top of ScaLAPACK. This approach is orthogonal (and complementary) to ours since it focuses on the heterogeneity of the processors [37], whereas we presently aim at mapping the implementation of the algorithm to the heterogeneity of the network (topology) through QCG-OMPI. In our present work, we do not consider the heterogeneity of the processors.…”

Section: A Dense Linear Algebra On the Gridmentioning

confidence: 99%

QR factorization of tall and skinny matrices in a grid computing environment

Agullo

Coti

Dongarra

et al. 2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing (IPDPS)

View full text Add to dashboard Cite

Previous studies have reported that common dense linear algebra operations do not achieve speed up by using multiple geographical sites of a computational grid. Because such operations are the building blocks of most scientific applications, conventional supercomputers are still strongly predominant in high-performance computing and the use of grids for speeding up large-scale scientific problems is limited to applications exhibiting parallelism at a higher level. We have identified two performance bottlenecks in the distributed memory algorithms implemented in ScaLAPACK, a state-of-the-art dense linear algebra library. First, because ScaLAPACK assumes a homogeneous communication network, the implementations of ScaLAPACK algorithms lack locality in their communication pattern. Second, the number of messages sent in the ScaLAPACK algorithms is significantly greater than other algorithms that trade flops for communication. In this paper, we present a new approach for computing a QR factorization -one of the main dense linear algebra kernels -of tall and skinny matrices in a grid computing environment that overcomes these two bottlenecks. Our contribution is to articulate a recently proposed algorithm (Communication Avoiding QR) with a topology-aware middleware (QCG-OMPI) in order to confine intensive communications (ScaLAPACK calls) within the different geographical sites. An experimental study conducted on the Grid'5000 platform shows that the resulting performance increases linearly with the number of geographical sites on large-scale problems (and is in particular consistently higher than ScaLAPACK's).

show abstract

Section: A Dense Linear Algebra On the Gridmentioning

confidence: 99%

QR factorization of tall and skinny matrices in a grid computing environment

Agullo

Coti

Dongarra

et al. 2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing (IPDPS)

View full text Add to dashboard Cite

show abstract

“…Efficient implementations of scientific codes are achieved in many cases by the efficient use of those routines, and it is necessary to study their behaviour in the systems where they are run. The basic component of BLAS and of many of the scientific codes is the matrix multiplication, and the efficient development of other linear algebra packages (LAPACK [3], ScaLAPACK [4], PLA-PACK [5], HeteroScaLAPACK [6], etc.) is directly based on that of the matrix multiplication routine.…”

Section: Introductionmentioning

confidence: 99%

Improving Linear Algebra Computation on NUMA Platforms through Auto-tuned Nested Parallelism

Cuenca

García

Giménez

2012

2012 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing

View full text Add to dashboard Cite

The most computationally demanding scientific and engineering problems are solved with large parallel systems. In some cases those systems are Non-Uniform Memory Access multiprocessors made up of a large number of cores which share a hierarchically organized memory. Basic linear algebra routines of the type of BLAS typically constitute the kernel of the computation for those problems, and the efficient use of these routines in those systems would contribute to a faster solution of a large range of scientific problems. Normally some multithreaded BLAS library optimized for the system is used, but when the number of cores increases the degradation in the performance is significant, and this can produce a misuse of the large, expensive systems. This paper empirically analyses the behaviour in large NUMA systems of the matrix multiplication of the BLAS library, and its combination with OpenMP to obtain nested parallelism. With the auto-tuning method proposed in this work, a reduction in the execution time is achieved with respect to the matrix multiplication of the library.

show abstract

“…Both mpC and HeteroMPI have been used for development of a wide range of real-life applications. HeteroMPI was also the instrumental tool for implementation of Heterogeneous ScaLAPACK [42], a version of ScaLAPACK optimized for heterogeneous clusters of workstations.…”

Section: Programming Toolsmentioning

confidence: 99%