Selecting representative benchmark inputs for exploring microprocessor design spaces

Abstract

The design process of a microprocessor requires representative workloads to steer the search process toward an optimum design point for the target application domain. However, considering a broad set of workloads to cover the large space of potential workloads is infeasible given how time-consuming design space exploration typically is. Hence, it is crucial to select a small yet representative set of workloads, which leads to a shorter design cycle while yielding a (near) optimal design.Prior work has mostly looked into selecting representative benchmarks; however, limited attention was given to the selection of benchmark inputs and how this affects workload representativeness during design space exploration. Using a set of 1,000 inputs for a number of embedded benchmarks and a design space with around 1,700 design points, we find that selecting a single or three random input(s) per benchmark potentially (in a worst-case scenario) leads to a suboptimal design that is 56% and 33% off, on average, relative to the optimal design in our design space in terms of Energy-Delay Product (EDP). We then propose and evaluate a number of methods for selecting representative inputs and show that we can find the optimum design point with as few as three inputs.

Evaluating Auto-Vectorizing Compilers through Objective Withdrawal of Useful Information

“…Another approach for automatically generating benchmarks is discussed in Deshpande et al [8], however, their focus is not in evaluating the vectorization capabilities of compilers. Skadron et al [29] highlight a number of practical issues in the current generation of benchmarking infrastructures, and Breughe and Eeckhout [4] propose a method for understanding the impact that different input parameters have on the overall performance. Gong et al [15] study the variations of compiler performance to different loop mutations, and in doing so they demonstrate the instability of compilers to consistently provide optimal results when the implementation is slightly different.…”