Statistical Models for Empirical Search-Based Performance Tuning

Vuduc, Richard; Demmel, James; Bilmes, Jeff

doi:10.1177/1094342004041293

Cited by 80 publications

(49 citation statements)

References 84 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vuduc et al [17] forward compiler analysis results to the runtime in the form of a decision function, in order to select among several versions of the same algorithm depending on input features. However, their optimization a ects program-and algorithm-speci c decision making during execution time, while we focus on general runtime system decisions made at compile time.…”

Section: Related Workmentioning

confidence: 99%

Task-parallel Runtime System Optimization Using Static Compiler Analysis

Thoman

Zangerl

Fahringer

2017

Proceedings of the Computing Frontiers Conference

View full text Add to dashboard Cite

Achieving high performance in task-parallel runtime systems, especially with high degrees of parallelism and ne-grained tasks, requires tuning a large variety of behavioral parameters according to program characteristics. In the current state of the art, this tuning is generally performed in one of two ways: either by a group of experts who derive a single setup which achieves good -but not optimal -performance across a wide variety of use cases, or by monitoring a system's behavior at runtime and responding to it. e former approach invariably fails to achieve optimal performance for programs with highly distinct execution pa erns, while the la er induces some overhead and cannot a ect parameters which need to be xed at compile time.In order to mitigate these drawbacks, we propose a set of novel static compiler analyses speci cally designed to determine program features which a ect the optimal se ings for a task-parallel execution environment. ese features include the parallel structure of task spawning, the granularity of individual tasks, and an estimate of the stack size required per task. Based on the result of these analyses, various runtime system parameters are then tuned at compile time.We have implemented this approach in the Insieme compiler and runtime system, and evaluated its e ectiveness on a set of 12 task parallel benchmarks running with 1 to 64 hardware threads. Across this entire space of use cases, our implementation achieves a geometric mean performance improvement of 39%.

show abstract

Section: Related Workmentioning

confidence: 99%

Task-parallel Runtime System Optimization Using Static Compiler Analysis

Thoman

Zangerl

Fahringer

2017

Proceedings of the Computing Frontiers Conference

View full text Add to dashboard Cite

show abstract

“…Iterative compilation has been shown to regularly outperform the most aggressive compilation settings of commercial compilers, and it has often been comparable to hand-optimized library functions [39], [16], [33], [38].…”

Section: Related Workmentioning

confidence: 99%

Predictive Modeling in a Polyhedral Optimization Space

Park

Cavazos²,

Pouchet³

et al. 2013

Int J Parallel Prog

View full text Add to dashboard Cite

Abstract-Significant advances in compiler optimization have been made in recent years, enabling many transformations such as tiling, fusion, parallelization and vectorization on imperfectly nested loops. Nevertheless, the problem of finding the best combination of loop transformations remains a major challenge. Polyhedral models for compiler optimization have demonstrated strong potential for enhancing program performance, in particular for compute-intensive applications. But existing static cost models to optimize polyhedral transformations have significant limitations, and iterative compilation has become a very promising alternative to these models to find the most effective transformations. But since the number of polyhedral optimization alternatives can be enormous, it is often impractical to iterate over a significant fraction of the entire space of polyhedrally transformed variants. Recent research has focused on iterating over this search space either with manually-constructed heuristics or with automatic but very expensive search algorithms (e.g., genetic algorithms) that can eventually find good points in the polyhedral space.In this paper, we propose the use of machine learning to address the problem of selecting the best polyhedral optimizations. We show that these models can quickly find high-performance program variants in the polyhedral space, without resorting to extensive empirical search. We introduce models that take as input a characterization of a program based on its dynamic behavior, and predict the performance of aggressive high-level polyhedral transformations that includes tiling, parallelization and vectorization. We allow for a minimal empirical search on the target machine, discovering on average 83% of the searchspace-optimal combinations in at most 5 runs. Our end-to-end framework is validated using numerous benchmarks on two multi-core platforms.

show abstract

“…Vuduc et al construct statistical learning models to build different decision functions for matrix-matrix multiplication algorithm selection [13]. In their work, they consider three methods for decision function construction: parametric modeling; parametric geometry modeling; and non-parametric geometry modeling.…”

Section: Related Workmentioning

confidence: 99%

Decision Trees and MPI Collective Algorithm Selection Problem

Pjesivac-Grbovic

Bosilca

Fagg

et al. 2007

Euro-Par 2007 Parallel Processing

View full text Add to dashboard Cite

Selecting the close-to-optimal collective algorithm based on the parameters of the collective call at run time is an important step in achieving good performance of MPI applications. In this paper, we explore the applicability of C4.5 decision trees to the MPI collective algorithm selection problem. We construct C4.5 decision trees from the measured algorithm performance data and analyze the decision tree properties and expected run time performance penalty.In cases we considered, results show that the C4.5 decision trees can be used to generate a reasonably small and very accurate decision function. For example, the Broadcast decision tree with only 21 leaves was able to achieve a mean performance penalty of 2.08%. Similarly, combining experimental data for Reduce and Broadcast and generating a decision function from the combined decision trees resulted in less than 2.5% relative performance penalty. The results indicate that C4.5 decision trees are applicable to this problem and should be more widely used in this domain.

show abstract

Statistical Models for Empirical Search-Based Performance Tuning

Cited by 80 publications

References 84 publications

Task-parallel Runtime System Optimization Using Static Compiler Analysis

Task-parallel Runtime System Optimization Using Static Compiler Analysis

Predictive Modeling in a Polyhedral Optimization Space

Decision Trees and MPI Collective Algorithm Selection Problem

Contact Info

Product

Resources

About