HPC Benchmarking: Scaling Right and Looking Beyond the Average

Abstract

Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.

Memory bandwidth and latency in HPC: system requirements and performance impact

“…The UEABS applications mimic large-scale production HPC workloads; they have large input data-sets and can scale up to thousands of processes. Extracting the UEABS execution segments representative of the production runs is not trivial [139] sphinx3 [60] leslie3d [48] libquantum [36] lbm [32] omnetpp [64] soplex [36] GemsFDTD [18] milc [20] cactusADM [20] gcc [24] astar [48] hmmer [64] zeusmp [30] xalancbmk [36] bwaves [18] wrf [22] dealII [24] h264ref [64] bzip2 [18] mcf [8] gobmk [64] gromacs [64] sjeng [64] perlbench [26] tonto [44] namd [64] calculix [48] gamess [48] povray [ tion tools, such as Limpio, Extrae and Paraver, all of which were used in this study and described in Section 3.3.1. We used a Yokogawa WT230 [140] power meter to measure the server power consumption.…”