Using Machine Learning to Focus Iterative Optimization

Agakov, Felix; Bonilla, Edwin V.; Cavazos, John; Franke, Björn; Fursin, Grigori; O’Boyle, Michael; Thomson, John; Toussaint, Michel; Williams, Christopher K. I.

doi:10.1109/cgo.2006.37

Cited by 296 publications

(348 citation statements)

References 13 publications

Supporting

Mentioning

341

Contrasting

Unclassified

Order By: Relevance

“…Within the scope of this survey, scientific publications that use machine learning for code optimization at compile time include [1,17,34,35,57,69,87,95,99], whereas scientific publications that use meta-heuristics for code optimization include [21,88,92,93]. Table 4 lists the characteristics of the selected primary studies that address code optimization at compile time.…”

Section: Code Optimizationmentioning

confidence: 99%

“…With regards to the machine learning algorithms used for code optimization, Nearest Neighbor (NN) classifier [1,57,87,99], Support Vector Machine (SVM) [87,95], and Decision Tree (DT) [69] are the most popular. Other algorithms, such as Ruled Set Induction (RSI) [17], and Predictive Search Distribution (PSD) [34,35] are also used for code optimization during compilation.…”

Section: Rq2: Software Optimization Algorithms Used For Compile-time mentioning

confidence: 99%

“…Uses profiling to detect loop candidates, which may significantly increase the compilation time [1,21] Adaptive tuning of the compilation process Profiling data needs to be collected to perform the virtual executions. Takes too long to find the optimal transformations.…”

Section: Meta-heuristicsmentioning

confidence: 99%

See 2 more Smart Citations

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

et al. 2018

View full text Add to dashboard Cite

While modern parallel computing systems offer high performance, utilizing these powerful computing resources to the highest possible extent demands advanced knowledge of various hardware architectures and parallel programming models. Furthermore, optimized software execution on parallel computing systems demands consideration of many parameters at compile-time and run-time. Determining the optimal set of parameters in a given execution context is a complex task, and therefore to address this issue researchers have proposed different approaches that use heuristic search or machine learning. In this paper, we undertake a systematic literature review to aggregate, analyze and classify the existing software optimization methods for parallel computing systems. We review approaches that use machine learning or meta-heuristics for software optimization at compile-time and run-time. we discuss challenges and future research directions. The results of this study may help to better understand the state-of-the-art techniques that use machine learning and meta-heuristics to deal with the complexity of software optimization for parallel computing systems. Furthermore, it may aid in understanding the limitations of existing approaches and identification of areas for improvement.

show abstract

Section: Code Optimizationmentioning

confidence: 99%

Section: Rq2: Software Optimization Algorithms Used For Compile-time mentioning

confidence: 99%

See 1 more Smart Citation

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Agakov et al [3] and Cavazos et al [8] use machine-learning to focus iterative search using either syntactic program features or dynamic hardware counters and multiple program transformations. Most of these works also require a large number of training runs.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Over the years, the approach has been perfected with fast optimization space search techniques, sophisticated machine-learning algorithms and continuous optimization [25,29,28,34,3,8,32,23,21]. And, even though these different research works have demonstrated significant performance improvements, the technique is far from mainstream in production environments.…”

Section: Introductionmentioning

confidence: 99%

Collective Optimization

Fursin

Temam

2009

High Performance Embedded Architectures and Compilers

View full text Add to dashboard Cite

Abstract. Iterative compilation is an efficient approach to optimize programs on rapidly evolving hardware, but it is still only scarcely used in practice due to a necessity to gather a large number of runs often with the same data set and on the same environment in order to test many different optimizations and to select the most appropriate ones. Naturally, in many cases, users cannot afford a training phase, will run each data set once, develop new programs which are not yet known, and may regularly change the environment the programs are run on. In this article, we propose to overcome that practical obstacle using Collective Optimization, where the task of optimizing a program leverages the experience of many other users, rather than being performed in isolation, and often redundantly, by each user. Collective optimization is an unobtrusive approach, where performance information obtained after each run is sent back to a central database, which is then queried for optimizations suggestions, and the program is then recompiled accordingly. We show that it is possible to learn across data sets, programs and architectures in non-dynamic environments using static function cloning and run-time adaptation without even a reference run to compute speedups over the baseline optimization. We also show that it is possible to simultaneously learn and improve performance, since there are no longer two separate training and test phases, as in most studies. We demonstrate that extensively relying on competition among pairs of optimizations (program reaction to optimizations) provides a robust and efficient method for capturing the impact of optimizations, and for reusing this knowledge across data sets, programs and environments. We implemented our approach in GCC and will publicly disseminate it in the near future.

show abstract

References

2011

Computer System Design

View full text Add to dashboard Cite

Using Machine Learning to Focus Iterative Optimization

Cited by 296 publications

References 13 publications

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

Collective Optimization

References

Contact Info

Product

Resources

About