Performance modeling for systematic performance tuning

Abstract

The performance of parallel scientific applications depends on many factors which are determined by the execution environment and the parallel application. Especially on large parallel systems, it is too expensive to explore the solution space with series of experiments. Deriving analytical models for applications and platforms allow estimating and extrapolating their execution performance, bottlenecks, and the potential impact of optimization options. We propose to use such "performance modeling" techniques beginning from the application design process throughout the whole software development cycle and also during the lifetime of supercomputer systems. Such models help to guide supercomputer system design and re-engineering efforts to adopt applications to changing platforms and allow users to estimate costs to solve a particular problem. Models can often be built with the help of well-known performance profiling tools. We discuss how we successfully used modeling throughout the proposal, initial testing, and beginning deployment phase of the Blue Waters supercomputer system.

Using automated performance modeling to find scalability bugs in complex codes

“…Another metric analyzed was the number of floating-point operations in each invocation of the time-intensive kernels as a function of the number of grid points per process. The results in Table 3 show that the number of floating-point operations per kernel invocation is proportional to the number of grid points (rightmost column), which is again consistent with [16]. All kernels but the conjugate-gradient kernel (ks_congrad) have a constant number of invocations, whereas the number of times the conjugate-gradient kernel is invoked depends for this particular input matrix on the number of grid points (middle column).…”

“…Since there is no performance variation in these requirements measurements, the quality of the automated fit (and thus the confidence) is high, resulting in a model that matches the handcrafted counterpart exactly. Our method also found the number of messages in each kernel to be invariant regardless of the lattice size, which further matches the models in [16]. Another metric analyzed was the number of floating-point operations in each invocation of the time-intensive kernels as a function of the number of grid points per process.…”

“…via parallel simulations of the SU(3) lattice gauge theory on a four-dimensional lattice. In earlier work [16], analytical models were manually created that describe the behavior of MILC/su3 rmd, one of the MILC codes, by characterizing its most important components with respect to a number of parameters. We now show that our modeling tool chain allows similar models to be derived automatically.…”

“…Given that MILC is known to scale well, we refined the default setting for I by adding , as suggested in Section 2.3. We collected five data points for each function at the scales P3 = 2 7 , 2 8 , 2 9 , 2 10 , 2 11 , 2 12 , 2 13 , 2 14 , 2 15 , 2 16 with a local lattice size of V = 9 4 per process. All model functions generated for Juqueen are shown in Table 2.…”

Automatic complexity analysis of explicitly parallel programs

“…Partial execution [35] can improve those techniques. Other studies provide advice for modeling the general performance [21] and scalability [19] of parallel applications. In addition, many application-specific studies exist but cannot be generalized [7,24].…”