Programming with BSP Homomorphisms

Abstract

Abstract. Algorithmic skeletons in conjunction with list homomorphisms play an important role in formal development of parallel algorithms. We have designed a notion of homomorphism dedicated to bulk synchronous parallelism. In this paper we derive two application using this theory: sparse matrix vector multiplication and the all nearest smaller values problem. We implement a support for BSP homomorphism in the Orléans Skeleton Library and experiment it with these two applications.

Formal derivation and extraction of a parallel program for the all nearest smaller values problem

“…This formally derived version has been compared to an implementation of the same algorithm (obtained by pen-and-paper derivation) using the Orléans Skeleton Library (OSL) [14] for C++, and a direct implementation of the algorithm of He and Huang [12] in the C language using the MPI library. This less general version is more efficient but its verification would be a huge work.…”

“…Orléans Skeleton Library or OSL [14] is a library of algorithmic skeletons for C++, implemented on top of MPI and that uses meta-programming techniques to optimise the composition of skeletons. This library contains a set of classical data parallel skeletons and also the bh skeleton.…”

“…He and Huang were the first to publish a practical implementation together with experiments for a BSP version of an algorithm that solves the ANSV problem [12]. Recently it has been shown that the ANSV problem can be derived using the theory of BSP homomorphisms (pen-and-paper method) and implemented in parallel with the bh skeleton of the OSL library [14].…”

“…inefficient) functional program, a more efficient composition of higher-order functions is derived step-by-step using an equational theory in a penand-paper way. From this last functional expression a parallel program is implemented, very often using a skeleton library for an imperative language such as C++: Muesli [3], SkeTo [19], or OSL [14] to cite a few.…”

PySke: Algorithmic Skeletons for Python

“…Compared to OSL, PySke lacks a skeleton [23] to manage exceptions in parallel. OSL also provides bh a parallel parallel skeleton well-suited for bulk synchronous parallelism [24]. PySke does not provide this skeleton yet.…”