An introduction to compilation issues for parallel machines

Abstract

The exploitation of today's high-performance computer systems requires the effective use of parallelism in many forms and at numerous levels. This survey article discusses program analysis and restructuring techniques that target parallel architectures. We first describe various categories of architectures that are oriented toward parallel computation models: vector architectures, shared-memory multiprocessors, massively parallel machines, message-passing architectures, VLIWs, and multithreaded architectures. We then describe a variety of optimization techniques that can be applied to sequential programs to effectively utilize the vector and parallel processing units. After an overview of basic dependence analysis, we present restructuring transformations on DO loops targeted both to vectorization and to concurrent execution, interprocedural and pointer analysis, task scheduling, instruction-level parallelization, and compiler-assisted data placement. We conclude that although tremendous advances have been made in dependence theory and in the development of a "toolkit" of transformations, parallel systems are used most effectively when the programmer interacts in the optimization process.

A Survey of Loop Parallelization: Models, Approaches, and Recent Developments

“…A data dependence is induced by two adjacent accesses to a same memory variable, and at least one of them is a write [4,6]. For convenience, we call an execution of a statement as a statement instance.…”

HyperPascal for architecture independent parallel programming

“…This being the case, it very inefficient to pass frequent small messages between processors. Consequently, the compilation system should ensure that individual processors are assigned larger and relatively independent threads of control [20]. The parallelism is thus coarsegrained; often at the procedure or subtask level.…”

HyperPascal-an architecture independent Pascal interface for parallel programming

“…This being the case, it very inefficient t o pass frequent small messages between processors. Consequently, the compilation system should ensure that individual processors are assigned larger and relatively independent threads of control [20]. The parallelism is thus coarsegrained; often at the procedure or subtask level.…”