Architecture-Adaptive Code Variant Tuning

Abstract

Code variants represent alternative implementations of a computation, and are common in high-performance libraries and applications to facilitate selecting the most appropriate implementation for a specific execution context (target architecture and input dataset). Automating code variant selection typically relies on machine learning to construct a model during an offline learning phase that can be quickly queried at runtime once the execution context is known. In this paper, we define a new approach called architectureadaptive code variant tuning, where the variant selection model is learned on a set of source architectures, and then used to predict variants on a new target architecture without having to repeat the training process. We pose this as a multi-task learning problem, where each source architecture corresponds to a task; we use device features in the construction of the variant selection model. This work explores the effectiveness of multi-task learning and the impact of different strategies for device feature selection. We evaluate our approach on a set of benchmarks and a collection of six NVIDIA GPU architectures from three distinct generations. We achieve performance results that are mostly comparable to the previous approach of tuning for a single GPU architecture without having to repeat the learning phase.

A benchmark set of highly-efficient CUDA and OpenCL kernels and its dynamic autotuning with Kernel Tuning Toolkit

“…Online tuning is easier to implement than dynamic tuning (there is no runtime compilation), but it is not practical when the number of possible code variants is high. An examples of an online tuner is SOCRATES [17,33].…”

“…Machine learning on historical autotuning data can be used to decrease the number of tuning decisions performed during program compilation or execution. In [33], a dynamic selection from a very limited number of code variants is based on a model created from previous tuning runs. In [8], a single tuning parameter can be optimized at compilation time by a neural network trained in multiple trial runs.…”

Exploiting historical data: Pruning autotuning spaces and estimating the number of tuning steps

“…However, no historical data from previous tuning runs on different hardware has been used. Data from previous runs are used for learning by Muralidharan et al 21 and Cummins et al 22 Those papers focus on dynamic selection from a very limited number of code variants, 21 or optimization of a single tuning parameter at compilation time 22 . We are focusing on the usage of historical data in more complex tuning spaces.…”