Optimizing for reduced code space using genetic algorithms

Cooper, Keith D.; Schielke, Philip J.; Subramanian, Devika

doi:10.1145/315253.314414

Cited by 101 publications

(122 citation statements)

References 14 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…The difficulty of achieving portable performance has led to empirical iterative compilation for statically compiled programs [22,69,30,52,29,54,31,77,37,70,46,47,36,39,28,40], applying automatic compiler tuning based on feedback-directed compilation.…”

Section: Introductionmentioning

confidence: 99%

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Fursin

Kashnikov

Memon

et al. 2011

Int J Parallel Prog

218

251

View full text Add to dashboard Cite

Tuning compiler optimizations for rapidly evolving hardware makes porting and extending an optimizing compiler for each new platform extremely challenging. Iterative optimization is a popular approach to adapting programs to a new architecture automatically using feedback-directed compilation. However, the large number of evaluations required for each program has prevented iterative compilation from widespread take-up in production compilers. Machine learning has been proposed to tune optimizations across programs systematically but is currently limited to a few transformations, long training phases and critically lacks publicly released, stable tools.Our approach is to develop a modular, extensible, self-tuning optimization infrastructure to automatically learn the best optimizations across multiple programs and architectures based on the correlation between program features, run-time behavior and optimizations. In this paper we describe Milepost GCC, the first publicly-available open-source machine learning-based compiler. It consists of an Interactive Compilation Interface (ICI) and plugins to extract program features and exchange optimization data with the cTuning.org open public repository. It automatically adapts the internal optimization heuristic at function-level granularity to improve execution time, code size and compilation time of a new program on a given architecture. Part of the Milepost technology together with low-level ICI-inspired plugin framework is now included in the mainline GCC.We developed machine learning plugins based on probabilistic and transductive approaches to predict good combinations of optimizations. Our preliminary experimental results show that it is possible to automatically reduce the execution time of individual MiBench programs, some by more than a factor of 2, while also improving compilation 1 INRIA Saclay, France (HiPEAC member) · 2 University of Versailles Saint Quentin en Yvelines, France · 3 IBM Haifa, Israel (HiPEAC member) · 4 CAPS Entreprise, France (HiPEAC member) · 5 ARC International, UK · 6 University of Edinburgh, UK (HiPEAC member) · 2 time and code size. On average we are able to reduce the execution time of the MiBench benchmark suite by 11% for the ARC reconfigurable processor. We also present a realistic multi-objective optimization scenario for Berkeley DB library using Milepost GCC and improve execution time by approximately 17%, while reducing compilation time and code size by 12% and 7% respectively on Intel Xeon processor.

show abstract

Section: Introductionmentioning

confidence: 99%

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Fursin

Kashnikov

Memon

et al. 2011

Int J Parallel Prog

218

251

View full text Add to dashboard Cite

show abstract

“…In recent years, considerable research has addressed iterative compilation and its benefits have been reported in several publications [22], [10], [11], [15], [19], [1]. Iterative compilation has been shown to regularly outperform the most aggressive compilation settings of commercial compilers, and it has often been comparable to hand-optimized library functions [39], [16], [33], [38].…”

Section: Related Workmentioning

confidence: 99%

“…These techniques are very expensive and therefore only effective when programs are extremely small, such as those used in embedded domains. Cooper et al [10] used genetic algorithms to address the compilation phase-ordering problem. They were concerned with finding "good" compiler optimization sequences that reduced code size.…”

Section: Related Workmentioning

confidence: 99%

Predictive Modeling in a Polyhedral Optimization Space

Park

Cavazos²,

Pouchet³

et al. 2013

Int J Parallel Prog

View full text Add to dashboard Cite

Abstract-Significant advances in compiler optimization have been made in recent years, enabling many transformations such as tiling, fusion, parallelization and vectorization on imperfectly nested loops. Nevertheless, the problem of finding the best combination of loop transformations remains a major challenge. Polyhedral models for compiler optimization have demonstrated strong potential for enhancing program performance, in particular for compute-intensive applications. But existing static cost models to optimize polyhedral transformations have significant limitations, and iterative compilation has become a very promising alternative to these models to find the most effective transformations. But since the number of polyhedral optimization alternatives can be enormous, it is often impractical to iterate over a significant fraction of the entire space of polyhedrally transformed variants. Recent research has focused on iterating over this search space either with manually-constructed heuristics or with automatic but very expensive search algorithms (e.g., genetic algorithms) that can eventually find good points in the polyhedral space.In this paper, we propose the use of machine learning to address the problem of selecting the best polyhedral optimizations. We show that these models can quickly find high-performance program variants in the polyhedral space, without resorting to extensive empirical search. We introduce models that take as input a characterization of a program based on its dynamic behavior, and predict the performance of aggressive high-level polyhedral transformations that includes tiling, parallelization and vectorization. We allow for a minimal empirical search on the target machine, discovering on average 83% of the searchspace-optimal combinations in at most 5 runs. Our end-to-end framework is validated using numerous benchmarks on two multi-core platforms.

show abstract

“…[33,9,6,24,10,20,31,27,26,19] demonstrated that optimizations search techniques can effectively improve performance of statically compiled programs on rapidly evolving architectures, thereby outperforming state-of-the-art compilers, albeit at the cost of a large number of exploration runs.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Many recent research efforts have shown how iterative compilation can outperform static compiler optimizations and quickly adapt to complex processor architectures [33,9,6,24,16,10,20,31,27,26,18,19]. Over the years, the approach has been perfected with fast optimization space search techniques, sophisticated machine-learning algorithms and continuous optimization [25,29,28,34,3,8,32,23,21].…”

Section: Introductionmentioning

confidence: 99%

Collective Optimization

Fursin

Temam

2009

High Performance Embedded Architectures and Compilers

View full text Add to dashboard Cite

Abstract. Iterative compilation is an efficient approach to optimize programs on rapidly evolving hardware, but it is still only scarcely used in practice due to a necessity to gather a large number of runs often with the same data set and on the same environment in order to test many different optimizations and to select the most appropriate ones. Naturally, in many cases, users cannot afford a training phase, will run each data set once, develop new programs which are not yet known, and may regularly change the environment the programs are run on. In this article, we propose to overcome that practical obstacle using Collective Optimization, where the task of optimizing a program leverages the experience of many other users, rather than being performed in isolation, and often redundantly, by each user. Collective optimization is an unobtrusive approach, where performance information obtained after each run is sent back to a central database, which is then queried for optimizations suggestions, and the program is then recompiled accordingly. We show that it is possible to learn across data sets, programs and architectures in non-dynamic environments using static function cloning and run-time adaptation without even a reference run to compute speedups over the baseline optimization. We also show that it is possible to simultaneously learn and improve performance, since there are no longer two separate training and test phases, as in most studies. We demonstrate that extensively relying on competition among pairs of optimizations (program reaction to optimizations) provides a robust and efficient method for capturing the impact of optimizations, and for reusing this knowledge across data sets, programs and environments. We implemented our approach in GCC and will publicly disseminate it in the near future.

show abstract

Optimizing for reduced code space using genetic algorithms

Cited by 101 publications

References 14 publications

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Milepost GCC: Machine Learning Enabled Self-tuning Compiler

Predictive Modeling in a Polyhedral Optimization Space

Collective Optimization

Contact Info

Product

Resources

About