Generating performance portable code using rewrite rules: from high-level functional expressions to high-performance OpenCL code

Abstract

Computers have become increasingly complex with the emergence of heterogeneous hardware combining multicore CPUs and GPUs. These parallel systems exhibit tremendous computational power at the cost of increased programming effort resulting in a tension between performance and code portability. Typically, code is either tuned in a low-level imperative language using hardware-specific optimizations to achieve maximum performance or is written in a high-level, possibly functional, language to achieve portability at the expense of performance.We propose a novel approach aiming to combine high-level programming, code portability, and high-performance. Starting from a high-level functional expression we apply a simple set of rewrite rules to transform it into a low-level functional representation, close to the OpenCL programming model, from which OpenCL code is generated. Our rewrite rules define a space of possible implementations which we automatically explore to generate hardware-specific OpenCL implementations. We formalize our system with a core dependently-typed λ-calculus along with a denotational semantics which we use to prove the correctness of the rewrite rules.We test our design in practice by implementing a compiler which generates high performance imperative OpenCL code. Our experiments show that we can automatically derive hardwarespecific implementations from simple functional high-level algorithmic expressions offering performance on a par with highly tuned code for multicore CPUs and GPUs written by experts.

Stateful dataflow multigraphs

“…However, true performance portability cannot be achieved with these standards, as optimized code/directives vastly differ on each platform (especially in the case of FPGAs). Other frameworks mentioned below [7,18,31,45,58,61,63] also support imperative and massively parallel architectures (CPUs, GPUs), where Halide and Tiramisu have been extended [62] to target FPGA kernels. As opposed to SDFGs, none of the above models were designed to natively support both load/store architectures and reconfigurable hardware.…”

“…As the SDFG provides general-purpose state machines with dataflow, all the above models can be fully represented within it, where SDFGs have the added benefit of encapsulating fine-grained data dependencies. [18,44,51,58,60,63] provide a fixed set of high-level program transformations, similar to those presented on SDFGs. In particular, Halide's schedules are by definition data-centric, and the same applies to polyhedral loop transformations in CHiLL.…”

Optimizing convolutional neural networks on embedded platforms with OpenCL

“…The community will be encouraged to share their implementations of the basic building blocks of CNNs: from high-level, platform-agnostic descriptions (e.g. as functional expressions [18] or programs in PENCIL [9]) to lowlevel, platform-specific kernels (as can be found in vendor- (Only the AlexNet layers that take more than 1 second to execute are shown.) optimized libraries).…”

C++ EDSL for parallel code generation

“…Dedicated FP languages were proposed like NOVA, from NVIDIA [6]. In a separate context rewrite rules were investigated to generate low level representations of high-level parallel constructs [7]. Our approach is different from these works in the following way: first, these languages are inaccessible for the mainstream scientist, who are familiar with C/C++, and where only low level APIs are available.…”