A Performance Prediction Model for Memory-Intensive GPU Kernels

Abstract

Commodity graphic processing units (GPUs) have rapidly evolved to become high performance accelerators for data-parallel computing through a large array of processing cores and the CUDA programming model with a C-like interface. However, optimizing an application for maximum performance based on the GPU architecture is not a trivial task for the tremendous change from conventional multi-core to the manycore architectures. Besides, the GPU vendors do not disclose much detail about the characteristics of the GPU's architecture.To provide insights into the performance of memory-intensive kernels, we propose a pipelined global memory model to incorporate the most critical global memory performance related factor, uncoalesced memory access pattern, and provide a basis for predicting performance of memory-intensive kernels. As we will demonstrate, the pipeline throughput is dynamic and sensitive to the memory access patterns. We validated our model on the NVIDIA GPUs using CUDA (Compute Unified Device Architecture). The experiment results show that the pipeline captures performance factors related to global memory and is able to estimate the performance for memory-intensive GPU kernels via the proposed model.

Parallelizing unit test execution on GPU

“…Also, array access affects how memory is accessed, which is critical for performance diagnosis [44]. The performance of memory in GPUs relies on the memory access pattern that describes how data indexes are referred to by consecutive threads within a warp [33]. For example, coalesced memory access improves the performance of a program as a warp will not access multiple memory transactions [33].…”

“…The performance of memory in GPUs relies on the memory access pattern that describes how data indexes are referred to by consecutive threads within a warp [33]. For example, coalesced memory access improves the performance of a program as a warp will not access multiple memory transactions [33]. Also, if the cache is not aligned in a CUDA program, it affects the performance by 32 times more than cache-aligned [54].…”