Reproducible Experiments in Parallel Computing: Concepts and Stencil Compiler Benchmark Study

Abstract

For decades, the majority of the experiments on parallel computers have been reported at conferences and in journals usually without the possibility to verify the results presented. Thus, one of the major principles of science, reproducible results as a kind of correctness proof, has been neglected in the field of experimental high-performance computing. While this is still the state-of-the-art, current research targets for solutions to this problem. We discuss early results regarding reproducibility from a benchmark case study we did. In our experiments we explore the class of stencil calculations that are part of many scientific kernels and compare the performance results of four stencil compilers. In order to make these experiments reproducible from remote, a first prototype of an replication engine has been developed that can be accessed via the internet.

Autotuning Stencil Computations with Structural Ordinal Regression Learning

“…While PATUS has been used to target different multi-core CPUs and a single GPU, Halide is particularly focused on image processing and their execution on heterogeneous GPU+CPU platforms. PATUS also includes other stochastic and heuristic search techniques, and has shown its strength in tuning earthquake simulation applications [37] and reproducibility studies [38]. An improved GA was also used for tuning stencil computations targeting GPUs [16].…”

Trusted High-Performance Computing in the Classroom

“…sis, and visualization as executable components throughout the scientific exploration process [6]. These kinds of tools are not suitable to our case, since a common drawback is that they affect the performance measurements [7].…”