A GPGPU compiler for memory optimization and parallelism management

Abstract

This paper presents a novel optimizing compiler for general purpose computation on graphics processing units (GPGPU). It addresses two major challenges of developing high performance GPGPU programs: effective utilization of GPU memory hierarchy and judicious management of parallelism.The input to our compiler is a naïve GPU kernel function, which is functionally correct but without any consideration for performance optimization. The compiler analyzes the code, identifies its memory access patterns, and generates both the optimized kernel and the kernel invocation parameters. Our optimization process includes vectorization and memory coalescing for memory bandwidth enhancement, tiling and unrolling for data reuse and parallelism management, and thread block remapping or address-offset insertion for partition-camping elimination. The experiments on a set of scientific and media processing algorithms show that our optimized code achieves very high performance, either superior or very close to the highly fine-tuned library, NVIDIA CUBLAS 2.2, and up to 128 times speedups over the naive versions. Another distinguishing feature of our compiler is the understandability of the optimized code, which is useful for performance analysis and algorithm refinement.

Addressing GPU On-Chip Shared Memory Bank Conflicts Using Elastic Pipeline

“…Recently we also see some work in automating such manual optimizations [30,31]. Such high-level optimizations from the software side are complementary to our Elastic Pipeline proposal, albeit with the limitation discussed in Sect.…”

Optimistic Parallelism on GPUs

“…The memory accesses to these arrays will be monitored at runtime for misspeculation check. Although the compiler can identify the arrays that have irregular access patterns [18], not all of them will cause crossiteration dependence at runtime. Programmers can better identify which arrays need to be monitored.…”

“…Baskaran et al [3] use a polyhedral compiler model to optimize affine memory accesses in regular loops. Yang et al [18] presented an optimizing compiler for memory bandwidth enhancement, data reuse, parallelism management, etc.…”

“…The presence of dynamic irregularities prevents existing techniques from parallelizing the loops for GPUs. Therefore optimizing performance in their presence has been widely studied [20,3,9,18]. In this work, we consider a new class of dynamic irregularities in loops that may cause cross-iteration dependences at runtime.…”

Model-Driven Tile Size Selection for DOACROSS Loops on GPUs

“…Recent research on developing programming and compiler techniques for GPUs focuses on (among others) general programming principles [5,9,14], cost modeling and analysis [1,15,3], automatic code generation [2,11], and performance tuning and optimization [4,6,12,19]. However, these research efforts are almost exclusively limited to DOALL loops.…”