Experimental Verification and Analysis of Dynamic Loop Scheduling in Scientific Applications

Abstract

Scientific applications are often irregular and characterized by large computationally-intensive parallel loops. Dynamic loop scheduling (DLS) techniques can be used to improve the performance of computationally-intensive scientific applications via load balancing of their execution on high-performance computing (HPC) systems. Identifying the most suitable choices of data distribution strategies, system sizes, and DLS techniques which improve the performance of a given application, requires intensive assessment and a large number of exploratory native experiments (using real applications on real systems), which may not always be feasible or practical due to associated time and costs. In such cases, to avoid the execution of a large volume of exploratory native experiments, simulative experiments which are faster and less costly are more appropriate for studying the performance of applications for the purpose of optimizing it. This motivates the question of 'How realistic are the simulations of executions of scientific applications using DLS on HPC platforms?' In the present work, a methodology is devised to answer this question It involves the experimental verification and analysis of the performance of DLS in scientific applications. The proposed methodology is employed for a computer vision application executing using four DLS techniques on two different HPC platforms, both via native and simulative experiments. Moreover, the evaluation and the analysis of the native and simulative results indicate that the accuracy of the simulative experiments is strongly influenced by the values obtained by the chosen approach used to extract the computational effort of the application (FLOP-or time-based), the choice of application model representation into simulation (data or task parallel), and the choice of HPC subsystem models available in the simulator (multi-core CPUs, memory hierarchy, and network topology). Further insights into the native performance on two HPC platforms one versus the other, the simulated performance using the two SimGrid interfaces, one versus the other, and the native versus the simulated performance for each of the simulated HPC platforms, are also presented and discussed. The minimum and the maximum percent errors achieved between native and simulative experiments are 0.95% and 8.03%, respectively.

An approach for realistically simulating the performance of scientific applications on high performance computing systems

“…Several details of representing the application and the computing system characteristics in the simulation are presented and discussed, such as capturing the variability of native execution performance over multiple repetitions as well as calibrating and fine-tuning the simulated system representation for the execution of a specific application. The coupling between the application and the computing system representation has been shown to yield a very close agreement between the native and the simulative experimental results, and to achieve realistic simulative performance predictions [5].…”

“…The present work builds upon and extends own prior work [5] [6], which focused on the experimental verification of DLS implementation via reproduction [6] and the experimental verification of application's performance simulation on HPC systems [5], respectively. In the present work, a new method to represent the computational effort in tasks is explored and tested (c.f.…”

“…The FLOP count obtained with PAPI is used to represent the amount of work in each task in the simulation. The FLOP count per task is found to be a more accurate measurement to represent computational effort per task than time measurements as well as resulting in constant values across different application executions [5]. However, feeding the simulator the exact FLOP count per task might result in misrepresenting the dynamic behavior in native executions of tasks where their execution time varies among the different execution instances.…”

“…This comparison typically enables one to verify and justify the level of the agreement between the results of the native and the simulative experiments, and to answer the question of "How realistic are the simulations of applications performance on HPC systems?" [5].…”

SimAS: A simulation‐assisted approach for the scheduling algorithm selection under perturbations

“…To attain high trustworthiness in the performance results obtained with SG, the implementation of the nonadaptive DLS techniques in SG-SD has been verified [19] by reproducing the results presented in the work that introduced factoring [5]. The accuracy of the performance results obtained by simulative experiments against native experiments has recently also been quantified [20]. The present work employs the SG-SD interface to study the performance of scientific applications on a heterogeneous computing platform under perturbations.…”