Multicore/GPGPU Portable Computational Kernels via Multidimensional Arrays

Abstract

Large, complex scientific and engineering application code have a significant investment in computational kernels to implement their mathematical models. Porting these computational kernels to the collection of modern manycore accelerator devices is a major challenge in that these devices have diverse programming models, application programming interfaces (APIs), and performance requirements. The Trilinos-Kokkos array programming model provides librarybased approach to implement computational kernels that are performance-portable to CPU-multicore and GPGPU accelerator devices. This programming model is based upon three fundamental concepts: (1) there exists one or more manycore compute devices each with its own memory space, (2) data parallel kernels are executed via parallel for and parallel reduce operations, and (3) kernels operate on multidimensional arrays. Kernel execution performance is, especially for NVIDIA R GPGPU devices, extremely dependent on data access patterns. An optimal data access pattern can be different for different manycore devices -potentially leading to different implementations of computational kernels specialized for different devices. The Trilinos-Kokkos programming model support performance-portable kernels by separating data access patterns from computational kernels through a multidimensional array API. Through this API device-specific mappings of multiindices to device memory are introduced into a computational kernel through compile-time polymorphism; i.e., without modification of the kernel.

Kokkos Array performance-portable manycore programming model

“…A multi-index is simply an ordered list of integers denoted by (i 0 , i 1 , i 2 , · · · ). The rank of a multi-index is the number of indices; e.g., (1,3,5) and (7,5,3,1) are rank-3 and rank-4 multi-indices. A Kokkos Array multi-index space is a Cartesian project of integer ranges…”

“…Fundamental performance tests were applied in the early stages of Kokkos Array development [3], including comparison to hand-coded CUDA kernels. Results from these tests led to the following fundamental performance considerations.…”