Compiler transformations for high-performance computing

Abstract

In the last three decades a large number of compiler transformations for optimizing programs have been implemented. Most optimizations for uniprocessors reduce the number of instructions executed by the program using transformations based on the analysis of scalar quantities and data-flow techniques. In contrast, optimizations for high-performance superscalar, vector, and parallel processors maximize parallelism and memory locality with transformations that rely on tracking the properties of arrays using loop dependence analysis. This survey is a comprehensive overview of the important high-level program restructuring techniques for imperative languages, such as C and Fortran. Transformations for both sequential and various types of parallel architectures are covered in depth. We describe the purpose of each transformation, explain how to determine if it is legal, and give an example of its application. Programmers wishing to enhance the performance of their code can use this survey to improve their understanding of the optimizations that compilers can perform, or as a reference for techniques to be applied manually. Students can obtain an overview of optimizing compiler technology. Compiler writers can use this survey as a reference for most of the important optimizations developed to date, and as bibliographic reference for the details of each optimization. Readers are expected to be familiar with modern computer architecture and basic program compilation techniques.

Static Detection of Loop-Invariant Data Structures

“…In highperformance computing, loop optimizations play a key role in automated parallelization for exploiting the parallelism capabilities of the hardware. Broader overview and more detailed descriptions of these techniques is available from a number of sources (e.g., [12,13,14]). …”

Dynamic Look Ahead Compilation: A Technique to Hide JIT Compilation Latencies in Multicore Environment

“…An approach for fast hot spot estimation/detection is reported in [1]; another method-time-complexity evaluation can be found in [17]. Sophisticated techniques for optimization profit estimation are described in [26] and [4].…”

Automatic parallelization of irregular and pointer-based computations: Perspectives from logic and constraint programming

“…The relatively clean semantics of these languages also makes it comparatively easy to use formal methods and prove the transformations performed by the parallelizing compiler both correct and efficient. 3 Quite significant progress has been made in the past decade in the área of automatic program parallelization for logic programs and some of the challenges have been tackled quite effectively. In the following touch upon a few of them (see, for example, [11] for an overview of the área).…”

“…However, space limitations prevent us from considering these additional issues. 3 Functional programming is another paradigm which also facilitates exploitation of parallelism. However, it can be argued that the lack of certain features, such as pointers and backtracking, while making the parallelization problem easier, also precludes studying some interesting problems.…”

“…Some very significant progress has been made in parallelizing compilers for regular, numerical computations, generally based on the FORTRAN language (see, e.g., [3]). This research has resulted in well known concepts and techniques including a well understood notion of independence (based on the Bernstein conditions or, for example, more recent notions of "semantic independence" [4]), sophisticated syntactic loop transformations, transformations based on polytope models, extensive work on partitioning and placement, etc.…”