The tracefile testbed - a community repository for identifying and retrieving HPC performance data

Abstract

Background and motivationA high-performance computing (HPC) application is characterized by many variables that control its execution and determine its performance. Variables such as algorithm type, problem size, input parameters, programming languages and paradigms, libraries, hardware architecture, etc., can have very significant effects on program behavior. It is important to understand the role played by each variable and the ways they combine to influence the performance achieved, or achievable, by the application.Two approaches are commonly used for the purpose of understanding these effects: performance profiling and performance prediction. Profiling [7,8] captures the behavior of an application by monitoring its execution. Monitoring can be based on hardware counter sampling or it can require the instrumentation of the application's source code or its binary executable. The data produced by monitoring may be analyzed on-the-fly or stored as tracefiles for post-mortem analysis. Many of the tools currently available for HPC performance analysis are based on tracefiles. Examples include:• These techniques attempt to provide estimates of the performance achievable by an application by analyzing its structure and the influences of compiler transformations and the system architecture, using symbolic analysis, simulation, or other model-based methods. Prediction tools often rely directly or indirectly on tracefiles. The data from tracefiles can serve as the basis for constructing or validating the performance model, or can be used directly by the tool to adjust the model to the characteristics of a particular application (e.g., [9]).Tracefiles are typically generated by the application programmer as part of the performance tuning process. Our field studies of HPC programmers indicate that many experienced programmers also create suites of simple pseudo-benchmark codes and generate tracefiles to help establish basic performance characteristics when they