Performance prediction based on inherent program similarity

Abstract

A key challenge in benchmarking is to predict the performance of an application of interest on a number of platforms in order to determine which platform yields the best performance. This paper proposes an approach for doing this. We measure a number of microarchitecture-independent characteristics from the application of interest, and relate these characteristics to the characteristics of the programs from a previously profiled benchmark suite. Based on the similarity of the application of interest with programs in the benchmark suite, we make a performance prediction of the application of interest. We propose and evaluate three approaches (normalization, principal components analysis and genetic algorithm) to transform the raw data set of microarchitecture-independent characteristics into a benchmark space in which the relative distance is a measure for the relative performance differences. We evaluate our approach using all of the SPEC CPU2000 benchmarks and real hardware performance numbers from the SPEC website. Our framework estimates per-benchmark machine ranks with a 0.89 average and a 0.80 worst case rank correlation coefficient.

Selecting Computer Architectures by Means of Control-Flow-Graph Mining

“…A bit states if the respective subgraph is included in the CFG. 2 As the subgraphs in SG have a minimum size of one edge, single nodes are not included. We believe that these nodes provide important information as well.…”

“…Execution times for a number of programs are known for many systems, e.g., from benchmarks suites. [1,2] compare similarities of programs based on execution properties such as the CPU instruction mix. These properties are architecture-dependent, but independent of the implementation used.…”

“…The approach which probably is most similar to ours is [1,2]. Joshi et al use program characteristics to make statements on the similarities of programs [1].…”

“…Furthermore, generating the measures requires simulation or execution. Based on [1], Hoste et al use programsimilarity measures and predict performance with programs from the SPEC CPU 2000 benchmark suite [2]. They then normalise the microarchitectureindependent characteristics with techniques such as principal-component analysis.…”

“…To obtain enough reference points with known performance measures, the authors rely on programs from a benchmark suite. Like our approach, [2] can determine the best platform for an application. Its advantage is that predictions tend to be more accurate than ours.…”