Automatic Matching of Legacy Code to Heterogeneous APIs

Abstract

Heterogeneous accelerators often disappoint. They provide the prospect of great performance, but only deliver it when using vendor specific optimized libraries or domain specific languages. This requires considerable legacy code modifications, hindering the adoption of heterogeneous computing. This paper develops a novel approach to automatically detect opportunities for accelerator exploitation. We focus on calculations that are well supported by established APIs: sparse and dense linear algebra, stencil codes and generalized reductions and histograms. We call them idioms and use a custom constraint-based Idiom Description Language (IDL) to discover them within user code. Detected idioms are then mapped to BLAS libraries, cuSPARSE and clSPARSE and two DSLs: Halide and Lift. We implemented the approach in LLVM and evaluated it on the NAS and Parboil sequential C/C++ benchmarks, where we detect 60 idiom instances. In those cases where idioms are a significant part of the sequential execution time, we generate code that achieves 1.26x to over 20x speedup on integrated and external GPUs.

Progressive Raising in Multi-level IR

“…The first two categories have fallen out of fashion, mainly due to the weak robustness against code changes, leaving the stage for the more systematic semantic approaches. In this category, Ginsbach et al propose IDL an Idiom Description Language that gets lowered to a set of constraints [28]. A match is a code fragment that adheres to the set of specifications.…”

Code Detection for Hardware Acceleration Using Large Language Models

“…For existing code (e.g., a program written in a general-purpose language) there is a better alternative than rewriting the program using the acceleratorspecific language. This alternative involves replacing parts of the code with calls to the accelerator API [9], [13]. By utilizing libraries that target the accelerator API, this ap-proach allows the compilation of old code, effectively using the accelerator without the need for extensive code rewriting.…”

“…• Constraint-based matching: Also known as patternbased matching, consists in finding constraints and patterns in the code that can be matched into a previously defined set of constraints [29]. Idiom Description Language (IDL) [13] proposed a description language that allows the user to define constraints for detecting particular idioms. Idioms are later translated into a set of constraints over the LLVM IR [19], which are used by the compiler to match the corresponding code.…”

“…This technique has many advantages, like improving the performance of a hardcoded implementation or offloading compute-heavy tasks to accelerators automatically. Many works focused on this topic [9], [13], [23], [45], which rely either on constrained-based pattern matching or neural network-based code detection. Those code detection techniques are generally either brittle, unable to match complex code, or inefficient in recognizing code patterns.…”

Cinnamon: A Domain-Specific Language for Binary Profiling and Monitoring

“…Lift [64] specifies high-level algorithmic expressions that can be translated down to OpenCL implementations for different platforms. The Idiom Description Language (IDL) [54] describes idioms or patterns in programs so that legacy sequential codes can be optimized and targeted to use different optimized APIs and DSLs for linear algebra libraries, such as BLAS and stencil computation. Halide [65] is a DSL for writing image-processing pipelines that separates the concern of specifying computation and how the computation is performed (i.e., through tiling, vectorization, or parallelization).…”