Parameterized Diamond Tiling for Stencil Computations with Chapel parallel iterators

Abstract

Stencil computations figure prominently in the core kernels of many scientific computations, such as partial differential equation solvers. Parallel scaling of stencil computations can be significantly improved on multicore processors using advanced tiling techniques that include the time dimension, such as diamond tiling. Such techniques are difficult to include in general purpose optimizing compilers because of the need for inter-procedural pointer and array data-flow analysis, plus the need to tune scheduling strategies and tile size parameters for each pairing of stencil computation and machine.Since a fully automatic solution is problematic, we propose to provide parameterized space and time tiling iterators through libraries. Ideally, the execution schedule or tiling code will be expressed orthogonally to the computation. This supports code reuse, easier tuning, and improved programmer productivity. Chapel iterators provide this capability implicitly. We present an advanced, parameterized tiling approach that we have implemented using Chapel parallel iterators. We show how such iterators can be used by programmers in stencil computations with multiple spatial dimensions. We also demonstrate that these new iterators provide better scaling than a traditional data parallel schedule.

The swept rule for breaking the latency barrier in time advancing PDEs

“…In this aspect, it is inspired by the Communication Avoiding (CA) algorithm [1,2] by Demmel et al Unlike the CA algorithm, however, the swept rule does not incur redundant computation. The swept rule shares some similarity with diamond tiling for cache optimization [3,4,5].…”

TOAST: Automatic tiling for iterative stencil computations on GPUs

“…Automatic tiling approaches typically target CPUs and are made available via compilersor frameworks propose a framework for automatic tiling optimizations on CPU‐GPU hybrid systems.…”

“…Automatic tiling approaches typically target CPUs and are made available via compilers 29,31 or frameworks. 10,17 In particular, Gysi et al 17 propose a framework for automatic tiling optimizations on CPU-GPU hybrid systems. Their approach performs automatic loop tiling and fusion on stencil applications in hybrid CPU-GPU systems, based on a compile-time performance model.…”

“…Despite the potential of tiling to improve application performance, bringing those improvements to fruition entails nontrivial challenges for the application developer. For example, the developer must decide on an optimal tile size within a very large parameter search space …”

“…For example, the developer must decide on an optimal tile size within a very large parameter search space. 10 In particular, too large of a tile size may cause a thrashing effect on the GPU memory, whereas too small of a tile size could cause poor data reuse and excessive loop overheads. Moreover, the developer must determine the best ordering of loop nesting to achieve the best locality, which, for iterative applications, also requires determining the maximum number of loop iterations possible within each tile to ensure correctness.…”