Fast compiler optimisation evaluation using code-feature based performance prediction

Dubach, Christophe; Cavazos, John; Franke, Björn; Fursin, Grigori; O’Boyle, Michael; Temam, Olivier

doi:10.1145/1242531.1242553

Cited by 61 publications

(38 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In compiler research, the feature sets used for predictive models are often provided without explanation and rarely is the quality of those features evaluated. More commonly, an initial large, high dimensional candidate feature space is pruned via feature selection [3], or projected into a lower dimensional space [43,44]. FEAST employs a range of existing feature selection methods to select useful candidate features [45].…”

Section: Related Workmentioning

confidence: 99%

End-to-End Deep Learning of Optimization Heuristics

Cummins

Petoumenos

Wang

et al. 2017

2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT)

144

154

View full text Add to dashboard Cite

Accurate automatic optimization heuristics are necessary for dealing with the complexity and diversity of modern hardware and software. Machine learning is a proven technique for learning such heuristics, but its success is bound by the quality of the features used. These features must be hand crafted by developers through a combination of expert domain knowledge and trial and error. This makes the quality of the final model directly dependent on the skill and available time of the system architect.Our work introduces a better way for building heuristics. We develop a deep neural network that learns heuristics over raw code, entirely without using code features. The neural network simultaneously constructs appropriate representations of the code and learns how best to optimize, removing the need for manual feature creation. Further, we show that our neural nets can transfer learning from one optimization problem to another, improving the accuracy of new models, without the help of human experts.We compare the effectiveness of our automatically generated heuristics against ones with features hand-picked by experts. We examine two challenging tasks: predicting optimal mapping for heterogeneous parallelism and GPU thread coarsening factors. In 89% of the cases, the quality of our fully automatic heuristics matches or surpasses that of state-of-theart predictive models using hand-crafted features, providing on average 16% and 12% more performance with no human effort expended on designing features.

show abstract

Section: Related Workmentioning

confidence: 99%

End-to-End Deep Learning of Optimization Heuristics

Cummins

Petoumenos

Wang

et al. 2017

2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT)

144

154

View full text Add to dashboard Cite

show abstract

“…Previous work [13], [8] has proposed to model the optimization problem by characterizing a program using performance counters. We use a prediction model originally proposed by Cavazos et al [12], [7], but slightly adapted to support polyhedral primitives instead. We refer to it as a speedup predictor model.…”

Section: B Speedup Prediction Modelmentioning

confidence: 99%

Predictive Modeling in a Polyhedral Optimization Space

Park

Cavazos²,

Pouchet³

et al. 2013

Int J Parallel Prog

Self Cite

View full text Add to dashboard Cite

Abstract-Significant advances in compiler optimization have been made in recent years, enabling many transformations such as tiling, fusion, parallelization and vectorization on imperfectly nested loops. Nevertheless, the problem of finding the best combination of loop transformations remains a major challenge. Polyhedral models for compiler optimization have demonstrated strong potential for enhancing program performance, in particular for compute-intensive applications. But existing static cost models to optimize polyhedral transformations have significant limitations, and iterative compilation has become a very promising alternative to these models to find the most effective transformations. But since the number of polyhedral optimization alternatives can be enormous, it is often impractical to iterate over a significant fraction of the entire space of polyhedrally transformed variants. Recent research has focused on iterating over this search space either with manually-constructed heuristics or with automatic but very expensive search algorithms (e.g., genetic algorithms) that can eventually find good points in the polyhedral space.In this paper, we propose the use of machine learning to address the problem of selecting the best polyhedral optimizations. We show that these models can quickly find high-performance program variants in the polyhedral space, without resorting to extensive empirical search. We introduce models that take as input a characterization of a program based on its dynamic behavior, and predict the performance of aggressive high-level polyhedral transformations that includes tiling, parallelization and vectorization. We allow for a minimal empirical search on the target machine, discovering on average 83% of the searchspace-optimal combinations in at most 5 runs. Our end-to-end framework is validated using numerous benchmarks on two multi-core platforms.

show abstract

“…This model has been used in recent work [9,13,22] to predict good optimization sequences for various different compilers. We refer to this model as the speedup predictor because it takes as input a program's characterization (P) and an optimization sequence (O), and it outputs the predicted speedup over some baseline for that optimization sequence.…”

Section: Speedup Prediction Modelmentioning

confidence: 99%

“…Using a well-constructed machine learning model to choose optimizations for a specific program has repeatedly been shown to outperform the most aggressive optimization levels in open-source and commercial compilers [6,10,13,15,16,19,20,22,25,30]. However, to use machine learning effectively, it is critical to use expressive features that characterize programs well and that strongly correlate to beneficial optimization sequences for the target program.…”

Section: Introductionmentioning

confidence: 99%

Using graph-based program characterization for predictive modeling

Park

Cavazos

Álvarez

2012

Proceedings of the Tenth International Symposium on Code Generation and Optimization

Self Cite

View full text Add to dashboard Cite

Using machine learning has proven effective at choosing the right set of optimizations for a particular program. For machine learning techniques to be most effective, compiler writers have to develop expressive means of characterizing the program being optimized. The current state-of-the-art techniques for characterizing programs include using a fixed-length feature vector of either source code features extracted during compile time or performance counters collected when running the program. For the problem of identifying optimizations to apply, models constructed using performance counter characterizations of a program have been shown to outperform models constructed using source code features. However, collecting performance counters requires running the program multiple times, and this "dynamic" method of characterizing programs can be specific to inputs of the program. It would be preferable to have a method of characterizing programs that is as expressive as performance counter features, but that is "static" like source code features and therefore does not require running the program.In this paper, we introduce a novel way of characterizing programs using a graph-based characterization, which uses the program's intermediate representation and an adapted learning algorithm to predict good optimization sequences. To evaluate different characterization techniques, we focus on loop-intensive programs and construct prediction models that drive polyhedral optimizations, such as auto-parallelism and various loop transformation.We show that our graph-based characterization technique outperforms three current state-of-the-art characterization techniques found in the literature. By using the sequences predicted to be the best by our graph-based model, we achieved up to 73% of the speedup achievable in our search space for a particular platform, whereas we could only achieve up to 59% by other state-of-the-art techniques we evaluated.

show abstract

Fast compiler optimisation evaluation using code-feature based performance prediction

Cited by 61 publications

References 29 publications

End-to-End Deep Learning of Optimization Heuristics

End-to-End Deep Learning of Optimization Heuristics

Predictive Modeling in a Polyhedral Optimization Space

Using graph-based program characterization for predictive modeling

Contact Info

Product

Resources

About