Parallel performance prediction for numerical codes in a multi-cluster environment

Abstract

We propose a model for describing and predicting the performance of parallel numerical software on distributed memory architectures within a multi-cluster environment. The goal of the model is to allow reliable predictions to be made as to the execution time of a given code on a large number of processors of a given parallel system, and on a combination of systems, by only benchmarking the code on small numbers of processors. This has potential applications for the scheduling of jobs in a Grid computing environment where informed decisions about which resources to use in order to maximize the performance and/or minimize the cost of a job will be valuable. The methodology is built and tested for a particular class of numerical code, based upon the multilevel solution of discretized partial differential equations, and despite its simplicity it is demonstrated to be extremely accurate and robust with respect to both the processor and communications architectures considered. Furthermore, results are also presented which demonstrate that excellent predictions may also be obtained for numerical algorithms that are more general than the pure multigrid solver used to motivate the methodology. These are based upon the use of a practical parallel engineering code that is briefly described. The potential significance of this work is illustrated via two scenarios which consider a Grid user who wishes to use the available resources either (i) to obtain a particular result as quickly as possible, or (ii) to obtain results to different levels of accuracy.

Reliable performance prediction for multigrid software on distributed memory systems

“…In these cases the model severely under predicts the measured run times. Based upon further numerical experiments (see, for example, [24,25]), we conjecture that the simple linear model (Eq. (3)) used to describe the latency term, a(np), in Eq.…”

“…In particular we consider heterogeneous and multicore processor architectures, combined with different communication architectures, such as Myrinet and Fast Ethernet switching. Furthermore, unlike our previously published work [23][24][25]27], we demonstrate the effectiveness of our approach on practical engineering software [12,13], in addition to codes that are based upon slightly less complex mathematical models [11].…”

“…T comm in (1), is described here. It is based on the model presented in [25]. For convenience, we define as ''work per processor'' the memory required by each processor: this is because the computational work per processor in a multigrid code is proportional to the problem size assigned, and therefore to the associated memory required by each processor.…”

“…This paper represents the development, and maturing, of our initial efforts to find such a middle ground, which have appeared in [23][24][25][26][27]. The goal is to build predictions based upon the empirical approach, in order to allow a wide variety of codes to be incorporated with relative ease, but to avoid the need for detailed and expensive benchmarking of every target HPC architecture, or for expensive traces to be executed for each parallel code being considered.…”

Coordinated rescheduling of Bag‐of‐Tasks for executions on multiple resource providers

“…Techniques based on trace analyses have the benefit of being application independent, however, may have limitations when workloads are highly heterogeneous. Application profiling has also been vastly studied to predict execution times . Application profiling can generate run‐time predictions for multiple environments but usually requires application source code access.…”