Abstract expressionism for parallel performance

Abstract

Programming with abstract, mathematical expressions offers benefits including terser programs, easier communication of algorithms, ability to prove theorems about algorithms, increased parallelism, and improved programming productivity. Common belief is that higher levels of abstraction imply a larger semantic gap between the user and computer and, therefore, typically slower execution, whether sequential or parallel. In recent years, domain-specific languages have been shown to close this gap through sophisticated optimizations benefitting from domain-specific knowledge.In this paper, we demonstrate that the semantic gap can also be closed for non-domain-specific functional array languages, without requiring embedding of language-specific semantic knowledge into the compiler tool chain. We present a simple example of APL-style SaC programs, compiled into C-code that outperform equivalent C programs in both sequential and parallel (OpenMP) environments.We offer insights into abstract expressionist programming, by comparing the characteristics and performance of a numerical relaxation benchmark written in C99, C99 with OpenMP directives, scheduling code, and pragmas, and in SaC, a functional array language. We compare three algorithmic styles: if/then/else, handoptimized loop splitting, and an abstract, functional style whose roots lie in APL.We show that the SaC algorithms match or outperform serial C, and that the hand-optimized and abstract SaC styles generate identical code, and so have identical performance. Furthermore, parallel SaC variants also outperform the best OpenMP C variant by up to a third, with no SaC source code modifications. Preserving an algorithm's abstract expression during optimization opens the door to generation of radically different code for different architectures.

The key to a data parallel compiler

“…Bernecky [5] argues that the increased programmability and desirable human factors of APL-style array programming can be implemented with competitive performance relative to more traditional techniques. This suggests that programs written in this highly abstract style may not sufer the performance gap traditionally assumed to go with their desirable features.…”