A multi-objective auto-tuning framework for parallel codes

Abstract

Auto-tuning has become increasingly popular for optimizing non-functional parameters of parallel programs. The typically large search space requires sophisticated techniques to find well performing parameter values in a reasonable amount of time. Different parts of a program often perform best with different parameter values. We therefore subdivide programs into several regions, and try to optimize the parameter values for each of those regions separately as opposed to setting the parameter values globally for the entire program. As this enlarges the search space even further, we have to extend existing auto-tuning techniques in order to obtain good results. In this paper we introduce a novel enhancement to the RS-GDE3 algorithm which is used to explore the search space for auto-tuning programs with multiple regions regarding several objectives. We have implemented our auto-tuner using the Insieme compiler and runtime system. In comparison to a non-optimized parallel version of the tested programs, our novel approach achieves up to 7.6, 10.5, and 61.6 fold improvements for three tuned objectives wall time, energy consumption, and resource usage, respectively.

Task-parallel Runtime System Optimization Using Static Compiler Analysis

“…• An implementation of this approach within the Insieme compiler and runtime system [8], targeting a set of four runtime parameters.…”

Characterizing Performance and Cache Impacts of Code Multi-versioning on Multicore Architectures

“…In order to evaluate the properties of our parameter space we used a custom toolset based on the Insieme research compiler and runtime system [11]. Our goal was to create a configurable number of versions of a function -all with the same runtime properties such as code size and execution time.…”

A Region-Aware Multi-Objective Auto-Tuner for Parallel Programs

“…After the Pareto set for the whole program is computed, a single configuration for the entire program can be selected from the Pareto set, either manually or automatically. This approach differs from the one proposed in [12], which is based on computing an individual Pareto set for every single region in isolation, making this approach prone to the performance penalties described in Section VIII. Furthermore, computing a Pareto set independently for every region requires a decision making process for every single region.…”

“…Additionally, the auto-tuner can evaluate the performance of a configuration for all regions at once which makes it aware of eventual performance penalties caused by region interferences. In Section VI, we compare this version of the auto-tuner presented in [12], which we call RS-GDE3 Global, against the new version of this paper.…”