Mitigating the compiler optimization phase-ordering problem using machine learning

Abstract

Today's compilers have a plethora of optimizations to choose from, and the correct choice of optimizations can have a significant impact on the performance of the code being optimized. Furthermore, choosing the correct order in which to apply those optimizations has been a long standing problem in compilation research. Each of these optimizations interacts with the code and in turn with all other optimizations in complicated ways. Traditional compilers typically apply the same set of optimization in a fixed order to all functions in a program, without regard the code being optimized.Understanding the interactions of optimizations is very important in determining a good solution to the phaseordering problem. This paper develops a new approach that automatically selects good optimization orderings on a per method basis within a dynamic compiler. Our approach formulates the phase-ordering problem as a Markov process and uses a characterization of the current state of the code being optimized to creating a better solution to the phase ordering problem. Our technique uses neuro-evolution to construct an artificial neural network that is capable of predicting beneficial optimization ordering for a piece of code that is being optimized. We implemented our technique in Jikes RVM and achieved significant improvements on a set of standard Java benchmarks over a well-engineered fixed order.

Guide Automatic Vectorization by means of Machine Learning: A Case Study of Tensor Contraction Kernels

“…Using machine learning to improve compilers is not a new idea, and the literature already contains several works that apply support vector machine (SVM) [4], [14], nearest neighbor (NN) [13], [14], artificial neural networks (ANN) [3], [5], and logistic regression [9]. They however make different types of predictions: Stephenson and Amarasinghe [14], Park et al [4] and Agakov et al [13] determine parameters of code optimizations; Kulkarni et al [3] order optimization passes in the middle end; Pekhimenko and Brown [9] use machine learning to focus search algorithms.…”

“…They however make different types of predictions: Stephenson and Amarasinghe [14], Park et al [4] and Agakov et al [13] determine parameters of code optimizations; Kulkarni et al [3] order optimization passes in the middle end; Pekhimenko and Brown [9] use machine learning to focus search algorithms.…”

Fast Automatic Heuristic Construction Using Active Learning

“…Its great advantage is that it can adapt to changing platforms as it has no a priori assumptions about their behaviour but it is expensive to train. There are many studies showing it outperforms human based approaches [2,3,[26][27][28][29]. Prior work for machine learning in compilers, as being exemplified by MilePost GCC project [30], often uses random sampling or exhaustive search to collect training examples.…”

“…Whenever a new processor comes out, even if derived from a previous one, the optimizing heuristics need to be re-tuned for it. This is typically too much effort and so, in fact, most compilers are out of date [2].…”

“…There are many studies showing that machine learning outperforms human based approaches [2,3]. Recent work also illustrates that it can be used to automatically port across architecture spaces [4] and can find more appropriate ways of mapping program parallelism to various platforms [5].…”