Scalable Kernel Fusion for Memory-Bound GPU Applications

Wahib, Mohamed; Maruyama, Naoya

doi:10.1109/sc.2014.21

Cited by 75 publications

(61 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They provide basic performance models for the number of stencils to fuse into one tile focusing on (possibly unrolled) kernels that process only one stencil repeatedly and do not consider varying tiling and fusion strategies. Finally, Wahib et al [18] take arbitrary stencil graphs from larger scientific applications and present an analytical performance model for choosing an optimal execution strategy. Even though closely related, they limit themselves to kernel fusion using computation on-the-fly only considering shared memory and apply their work on NVIDIA GPUs only.…”

Section: Related Workmentioning

confidence: 99%

“…most of these schemes consider the optimization of a single stencil in isolation. Many applications, however, require nested stencils [18] that are applied in succession. The data dependencies of these nestings can form complex directed acyclic stencil graphs where multiple stencils need to be optimized in tandem to achieve highest performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modesto

Gysi

Grosser

Hoefler

2015

Proceedings of the 29th ACM on International Conference on Supercomputing

View full text Add to dashboard Cite

Code transformations, such as loop tiling and loop fusion, are of key importance for the efficient implementation of stencil computations. However, their direct application to a large code base is costly and severely impacts program maintainability. While recently introduced domain-specific languages facilitate the application of such transformations, they typically still require manual tuning or auto-tuning techniques to select the transformations that yield optimal performance. In this paper, we introduce MODESTO, a model-driven stencil optimization framework, that for a stencil program suggests program transformations optimized for a given target architecture. Initially, we review and categorize data locality transformations for stencil programs and introduce a stencil algebra that allows the expression and enumeration of different stencil program implementation variants. Combining this algebra with a compile-time performance model, we show how to automatically tune stencil programs. We use our framework to model the STELLA library and optimize kernels used by the COSMO atmospheric model on multi-core and hybrid CPU-GPU architectures. Compared to naive and expert-tuned variants, the automatically tuned kernels attain a 2.0-3.1x and a 1.0-1.8x speedup respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Modesto

Gysi

Grosser

Hoefler

2015

Proceedings of the 29th ACM on International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…The problem is resolved by the OEG generating heuristic that adds precedency for the kernel invoked first by the host. Another optimization is to add redundant instances for arrays having several kernels writing into them to relax dependencies (elaboration on this optimization in previous work [28]). After this stage in the transformation, a report is given to the programmer regarding changes that were done to the original code to optimize the two graphs.…”

Section: Ddg and Oegmentioning

confidence: 99%

“…We build on previous work [28], which took the following steps. First, we formulated the kernel fusion problem as a combinatorial optimization problem.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automated GPU Kernel Transformations in Large-Scale Production Stencil Applications

Wahib

Maruyama

2015

Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing

Self Cite

View full text Add to dashboard Cite

This paper proposes an end-to-end framework for automatically transforming stencil-based CUDA programs to exploit inter-kernel data locality. The CUDA-to-CUDA transformation collectively replaces the user-written kernels by auto-generated kernels optimized for data reuse. The transformation is based on two basic operations, kernel fusion and fission, and relies on a series of automated steps: gathering metadata, generating graphs expressing dependencies and precedency constraints, searching for optimal kernel fissions/fusions, and generation of optimized code. The framework is modeled to provide the flexibility required for accommodating different applications, allowing the programmer to monitor and amend the intermediate results of different phases of the transformation. We demonstrate the practicality and effectiveness of automatic transformations in exploiting exposed data localities using a variety of real-world applications with large codebases that contain dozens of kernels and data arrays. Experimental results show that the proposed end-to-end automated approach, with minimum intervention from the user, improved performance of six applications with speedups ranging between 1.12x to 1.76x.

show abstract

Accelerating explicit ODE methods on GPUs by kernel fusion

Korch

Werner

2018

Concurrency and Computation

View full text Add to dashboard Cite

Graphics processing units (GPUs) have a promising architecture for implementing highly parallel solution methods for systems of ordinary differential equations (ODEs). However, their high performance comes at the price of caveats such as small caches or wide SIMD. For ODE methods, optimizing the memory access pattern is often crucial. In this article, instead of considering only one specific method, we generalize the description of explicit ODE methods by using data flow graphs consisting of basic operations that are suitable to cover the types of computations occurring in all common explicit methods. After showing that the straightforward approach for processing the data flow graph by calling one kernel per basic operation is memory bound, we explain how the number of memory accesses can be reduced by the kernel fusion technique, which fuses several basic operations into one kernel. Moreover, we will present enabling transformations that allow additional fusions and thus can reduce the number of memory accesses even further. We apply these optimizations to three different classes of explicit ODE methods: embedded Runge-Kutta (RK) methods, parallel iterated RK (PIRK) methods, and peer methods.A detailed experimental evaluation on three modern GPUs showed speedups between 1.86 and 3.51 compared to unfused implementations.

show abstract

Scalable Kernel Fusion for Memory-Bound GPU Applications

Cited by 75 publications

References 22 publications

Modesto

Modesto

Automated GPU Kernel Transformations in Large-Scale Production Stencil Applications

Accelerating explicit ODE methods on GPUs by kernel fusion

Contact Info

Product

Resources

About