Performance Comparison of Parallel Programming Environments for Implementing AIAC Algorithms

Abstract

AIAC algorithms (Asynchronous Iterations Asynchronous Communications) are a particular class of parallel iterative algorithms. Their asynchronous nature makes them more efficient than their synchronous counterparts in numerous cases as has already been shown in previous works. The first goal of this article is to compare several parallel programming environments in order to see if there is one of them which is best suited to efficiently implement AIAC algorithms. The main criterion for this comparison consists in the performances achieved in a global context of grid computing for two classical scientific problems. Nevertheless, we also take into account two secondary criteria which are the ease of programming and the ease of deployment. The second goal of this study is to extract from this comparison the important features that a parallel programming environment must have in order to be suited for the implementation of AIAC algorithms.

Performance analysis of asynchronous Jacobi’s method implemented in MPI, SHMEM and OpenMP

“…To date there has been work on both the theoretical [3,4,7] and the practical [2,5,9] aspects of such algorithms. To reason about these algorithms we need to understand what drives the speed of their convergence, but existing results merely provide sufficient conditions for the algorithms to converge, and do not help us to answer some of questions arising in the use of asynchronous techniques in large, tightly coupled parallel systems of relevance to Exascale computing.…”

“…Instead, we are interested in developing asynchronous techniques that avoid this blocking behaviour by permitting processors to operate on whatever data they have, even if new data has not yet arrived from other processors. To date, there has been work on both the theoretical (Chazan and Miranker, 1960;Baudet, 1978;Bertsekas and Tsitsiklis, 1989) and the practical (Bull and Freeman, 1992;Bahi et al, 2006; de Jager and Bradley, 2010) aspects of such algorithms. To reason about these algorithms we need to understand what drives the speed of their convergence, but existing results merely provide sufficient conditions for the algorithms to converge, and do not help us to answer some of questions arising in the use of asynchronous techniques in large, tightly coupled parallel systems of relevance to Exascale computing.…”