Automatic feature generation for machine learning--based optimising compilation

Abstract

Recent work has shown that machine learning can automate and in some cases outperform handcrafted compiler optimisations. Central to such an approach is that machine learning techniques typically rely upon summaries or features of the program. The quality of these features is critical to the accuracy of the resulting machine learned algorithm; no machine learning method will work well with poorly chosen features. However, due to the size and complexity of programs, theoretically there are an infinite number of potential features to choose from. The compiler writer now has to expend effort in choosing the best features from this space. This article develops a novel mechanism to automatically find those features that most improve the quality of the machine learned heuristic. The feature space is described by a grammar and is then searched with genetic programming and predictive modelling. We apply this technique to loop unrolling in GCC 4.3.1 and evaluate our approach on a Pentium 6. On a benchmark suite of 57 programs, GCCs hard-coded heuristic achieves only 3% of the maximum performance available, whereas a state-of-the-art machine learning approach with hand-coded features obtains 59%. Our feature generation technique is able to achieve 76% of the maximum available speedup, outperforming existing approaches.

Developer and user-transparent compiler optimization for interactive applications

“…It searches for different combinations of code transformations and transformation parameters, evaluates their effect on performance, and at the end keeps the best performing binary. Multiple approaches exist to select optimization sequences: random, through genetic search, statistical models [41], or machine learned models [33]. Despite the significant performance benefits of iterative compilation, it has not been applied in a general way on mobile systems.…”

SoCodeCNN: Program Source Code for Visual CNN Classification Using Computer Vision Methodology

“…and each feature extraction requires 3 seconds for a human being then such program would consume 3000 seconds or 50 minutes for complete feature extraction so that those features could be further used in machine learning algorithm. Since feature extraction from program source-code using NLP is heavily utilized in compiler optimization and thus someone could argue that the field of compiler optimization has improved a lot in past couple of years [24], [25]. However, given the emergence of specialized frameworks and directives such as OpenMP, OpenCL, OpenVX, modern automated code feature extraction methods [1]- [4] are still lacking in pace in terms of accurately extracting such features in a completely automated manner.…”

A methodology correlating code optimizations with data memory accesses, execution time and energy consumption

“…In [46], loop tiling is applied with fixed tile sizes. In [29] [28] and [50], only loop unroll transformation is applied. As a consequence, a very large number of solutions is not tested.…”

“…The most promising automatic approach is iterative compilation where many different versions of the program are generated-executed by applying a set of compiler transformations, at all different combinations/sequences. However, iterative compilation is extremely expensive in terms of compilation time and therefore researchers and current compilers try to reduce it by using i) both iterative compilation and machine learning compilation techniques [29] [5] [54] [38], ii) both iterative compilation and genetic algorithms [26], iii) heuristics and empirical methods [12], iv) both iterative compilation and statistical techniques [17], v) exhaustive search [25]. However, by employing these approaches, the remaining exploration space of code optimizations, i.e., the set of all optimization configurations that have to be explored (optimization sets), is still so large that searching is impractical.…”