MulticoreBSP for C: A High-Performance Library for Shared-Memory Parallel Programming

Abstract

The bulk synchronous parallel (BSP) model, as well as parallel programming interfaces based on BSP, classically target distributed-memory parallel architectures. In earlier work, Yzelman and Bisseling designed a MulticoreBSP for Java library specifically for shared-memory architectures. In the present article, we further investigate this concept and introduce the new high-performance MulticoreBSP for C library. Among other features, this library supports nested BSP runs. We show that existing BSP software performs well regardless whether it runs on distributedmemory or shared-memory architectures, and show that applications in MulticoreBSP can attain high-performance results. The paper details implementing the Fast Fourier Transform and the sparse matrix-vector multiplication in BSP, both of which outperform state-of-the-art implementations written in other shared-memory parallel programming interfaces. We furthermore study the applicability of BSP when working on highly non-uniform memory access architectures.The bulk synchronous parallel (BSP) model [19], introduced by Valiant, describes a powerful abstraction of parallel computers. It enables the design of theoretically optimal parallel algorithms, and inspired many interfaces for parallel programming. The BSP model consists of three parts: (1) an abstraction of a parallel computer, (2) an abstraction of a parallel algorithm, and (3) a cost model. A BSP computer has p homogeneous processors, each one with access to local memory. They cannot access remote memory, but may communicate through a black-box network interconnect. Preparing the network for all-to-all communication while synchronising the p processors at the start and end of communication costs l units; sending a data word during the all-to-all communication costs g units. Measuring l and g in seconds does not directly relate to any work done; instead, if the speed r of each processor is measured in floating-point operations per second (flop/s), we express l and g in flops as well. The four parameters ( p, r, l, g) completely define a BSP computer.A BSP algorithm runs on a BSP computer and adheres to the Single Program, Multiple Data (SPMD) paradigm. Each BSP process consists of alternating computation and communication phases. During computation, each process executes sequential code and cannot communicate with other BSP processes; during communication all processes are involved in an all-to-all data interchange and cannot perform any computations. BSP synchronises all processors in-between phases. We define one superstep as one computation phase combined with the communication phase that directly follows it.This definition of a BSP computer and a BSP algorithm immediately leads to the BSP cost model. If the algorithm consists of T supersteps, and if process s has w (s) i

A Study of Integer Sorting on Multicores

“…All implementations undertaken in this work are in ANSI C; the same code need only be recompiled but does not need to be rewritten to work with three parallel, multiprocessing or multithreaded programming libraries: OpenMPI [29], MulticoreBSP [36], and BSPlib [19]. The source code is available through the author's web-page [18].…”

Bulk: A Modern C++ Interface for Bulk-Synchronous Parallel Programs

“…These images can be used to access the remote array. For example, xs(t) [5] = 3 puts the value 3 into the array element at index 5 of the local array at processor t. Furthermore, convenient syntax makes it easy to work with slices of coarrays. A basic slice for the element interval [start, end), i.e., including start but excluding end, is obtained using xs(t) [{start, end}].…”

“…BSPlib [2] describes a collection of a limited set of primitives which can be used for writing BSP programs in the C programming language. Libraries that implement the BSPlib standard include BSPonMPI [3] and MulticoreBSP for Java [4] and C [5]. Paderborn University BSP (PUB) [6] is an alternative BSP library that includes features not included in BSPlib such as subset synchronization and nonblocking collective operations.…”

Predicting the Performance of Parallel Computing Models Using Queuing System

“…Currently, based on the existing computing models, such as MapReduce [1], BSP [2] (Bulk Synchronous Parallel model), much performance modeling research [3][4] [5] has been carried out to deeply analyze the parallel computing framework. [3]described the default mathematical model for predicting the performance of BSP model.…”

“…[3]described the default mathematical model for predicting the performance of BSP model. The performance of MapReduce applications that run on cloud has been accurately predicted in [4] [5].…”