Assessing Application Efficiency and Performance Portability in Single-Source Programming for Heterogeneous Parallel Systems

Abstract

We analyze the performance portability of the skeleton-based, single-source multi-backend high-level programming framework SkePU across multiple different CPU–GPU heterogeneous systems. Thereby, we provide a systematic application efficiency characterization of SkePU-generated code in comparison to equivalent hand-written code in more low-level parallel programming models such as OpenMP and CUDA. For this purpose, we contribute ports of the STREAM benchmark suite and of a part of the NAS Parallel Benchmark suite to SkePU. We show that for STREAM and the EP benchmark, SkePU regularly scores efficiency values above 80% and in particular for CPU systems, SkePU can outperform hand-written code.

Toward a Better Performance Portability Metric

“…For example, the two common architectural efficiency baselines are the theoretical peak performance of the platform of interest and the practical roofline peak performance of the platform of interest. Application efficiency is a popular measure because it is simple and easy to use [7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22]. All that is required is to measure the achieved runtime of the application on the given platform, and then to calculate its fraction relative to the runtime of the fastest known implementation of the application on the same platform.…”

“…And so it can happen that immediately after we have published our research, a faster implementation is found which makes the results of our findings obsolete. Furthermore, according to recent studies on application efficiency, researchers always chose as the baseline performance the performance of the implementation that showed the best performance from three or four implementations studied in their current research and not from those known in the literature [7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22]. If we add the observation that different studies use different compilers, compiler options, and input sizes-and that the source codes are not always available-it is clear that this situation leads to non-uniformity and incoherence of the results, and difficulties in reproducing them.…”