HPC Benchmarking: Problem Size Matters

Marjanovic, Vladimir; Gracia, José; Glass, Colin W.

doi:10.1109/pmbs.2016.006

Cited by 9 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Marjanović et al [14] explore the impact of input data-set for three representative benchmarks: HPL, HPCG and High-performance Geometric Multigrid (HPGMG). They perform an analysis on six distinct HPC platforms at the node level, and perform scale-out analysis on one of the platforms.…”

Section: Discussionmentioning

confidence: 99%

HPC Benchmarking: Scaling Right and Looking Beyond the Average

Radulovic

Asifuzzaman

Carpenter

et al. 2018

Euro-Par 2018: Parallel Processing

View full text Add to dashboard Cite

Designing a balanced HPC system requires an understanding of the dominant performance bottlenecks. There is as yet no well established methodology for a unified evaluation of HPC systems and workloads that quantifies the main performance bottlenecks. In this paper, we execute seven production HPC applications on a production HPC platform, and analyse the key performance bottlenecks: FLOPS performance and memory bandwidth congestion, and the implications on scaling out. We show that the results depend significantly on the number of execution processes and granularity of measurements. We therefore advocate for guidance in the application suites, on selecting the representative scale of the experiments. Also, we propose that the FLOPS performance and memory bandwidth should be represented in terms of the proportions of time with low, moderate and severe utilization. We show that this gives much more precise and actionable evidence than the average.

show abstract

Section: Discussionmentioning

confidence: 99%

HPC Benchmarking: Scaling Right and Looking Beyond the Average

Radulovic

Asifuzzaman

Carpenter

et al. 2018

Euro-Par 2018: Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Therefore, we select several major hardware components of the supercomputer as factors for the scoring model. Major hardware components of the supercomputer such as processor, memory, interconnection networks, I/O system, and storage are considered based on several previous studies [4,12,27,28]. As described above, the performance of a factor must be objectively measurable.…”

Section: Selection and Identification Of Factorsmentioning

confidence: 99%

“…In the life sciences field, memory capacity and memory bandwidth limitations present considerable challenges as a significant amount of memory is needed for large-scale genetic analysis [7,8]. Consequently, HPL has been strongly criticized for low correlations with the operating performance of applications because the measurements of HPL do not reflect the characteristics of other important factors of supercomputers [9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

“…The High Performance Conjugate Gradients (HPCG) benchmark that complements HPL reflects the characteristics of data access patterns of current applications. However, HPCG exhibits a shortcoming in which benchmark results are greatly affected by actual memory bandwidth [11,12]. In addition, other supercomputer benchmarking projects for specific purposes are in progress, such as Graph500, which focuses on supercomputer performance measurements for data-intensive applications, and Deep500, which is intended for performance measurement of deep learning applications.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Composite Measures of Supercomputer Technology

Kim¹,

On²,

Koh³

et al. 2019

KSII TIIS

View full text Add to dashboard Cite

We have developed composite measures of supercomputer technology, reflecting various factors of supercomputers using Martino's scoring model. CPUs, accelerators, memory, interconnection networks, and power consumption are chosen as factors of the model. The weight values of the factors are derived based on a survey of 129 domestic and international experts. The measured values are then standardized to integrate measurement units of the factors in the model. This model has been applied to 50 supercomputers, and rank correlation analysis was performed using representative measures. As a consequence, the ranking drastically changes except for the 1st and 2nd supercomputers on the TOP500. In addition, the characteristics of memory and interconnection networks influence the ranking, and the results demonstrate that the proposed model has low correlations with HPL and HPCG but a high correlation with Green500. This indicates that power consumption is an important factor that has a significant effect on the measures of supercomputer technology. In addition, it is determined that the differences between the HPL ranking and the proposed model ranking are influenced by power consumption, CPU theoretical peak performance, and main memory bandwidth in order of significance. In conclusion, the composite measures proposed in this study are more suitable for comprehensively describing supercomputer technology than existing performance measures. The findings of this study are expected to support decision making related to management and policy in the procurement and operation of supercomputers.

show abstract

“…Since 1979, LINPACK benchmark (HPL) has been used to measure the performance of HPC systems and the TOP500 list has been accordingly updated to rank the world's most powerful supercomputer. However, there have been many opinions criticizing whether HPL is a good metric to measure modern HPC performance or not . One of the main criticisms is its low memory to flop (Byte/Flop) ratio.…”

Section: Introductionmentioning

confidence: 99%

HPCG and HPGMG benchmark tests on multiple program, multiple data (MPMD) mode on Blue Waters—A Cray XE6/XK7 hybrid system

Kwack

Bauer

2017

Concurrency and Computation

View full text Add to dashboard Cite

SummaryThe high-performance conjugate gradients (HPCG) and high-performance geometric multi-grid (HPGMG) benchmarks are alternatives to the traditional LINPACK benchmark (HPL) in measuring the performance of modern HPC platforms. We performed HPCG and HPGMG benchmark tests on a Cray XE6/XK7 hybrid supercomputer, Blue Waters at National Center for Supercomputing Applications (NCSA). The benchmarks were tested on CPU-based and GPUenabled nodes separately, and then we analyzed characteristic parameters that affect their performance. Based on our analyses, we performed HPCG and HPGMG runs in multiple program, multiple data (MPMD) mode in Cray Linux Environment in order to measure their hybrid performance on both CPU-based and GPU-enabled nodes. We observed and analyzed several performance issues during those tests. Based on lessons learned from this study, we provide recommendations about how to optimize science applications on modern hybrid HPC platforms. KEYWORDSGPU, heterogeneous computing, HPC benchmark, hybrid HPC platforms, MPMD mode | INTRODUCTIONSince 1979, LINPACK benchmark (HPL) 1 has been used to measure the performance of HPC systems and the TOP500 list has been accordingly updated to rank the world's most powerful supercomputer. However, there have been many opinions criticizing whether HPL is a good metric to measure modern HPC performance or not. [2][3][4][5] One of the main criticisms is its low memory to flop (Byte/Flop) ratio. Byte/Flop ratios of many applications in molecular dynamics, weather forecasting, astrophysics, particle physics, structural analysis, and fluid dynamics are between 10 −1 and 10, while the ratio of HPL benchmark is less than 10 −3 . Since 2014, HPC community has shared the high-performance conjugate gradients (HPCG) 3,6 and high-performance geometric multi-grid (HPGMG) 4,7 benchmark results in order to represent modern HPC performance for many science and engineering HPC applications, at least, in terms of Byte/Flop ratios.This study is an extension of our continuing effort 8 to find out appropriate benchmarks for modern HPCs. In this study, we employed HPCG and HPGMG benchmarks to measure the performance of the Blue Waters system located at National Center for Supercomputing Application (NCSA). Blue Waters 9,10 is a Cray XE6/XK7 hybrid supercomputer with 22 640 CPU-based XE6 nodes and 4228 GPU-enabled XK7 nodes. The XE6 dual-socket nodes are populated with 2 AMD Interlargos model 6276 CPU processors with a nominal clock speed of at least 2.3 GHz and 64 GB of physical memory, while the XK7 accelerator nodes are equipped with one Interlagos model 6276 CPU processor and one NVIDIA GK110 "Kepler" accelerator K20X with 32 GB of CPU memory and 6 GB of GPU memory. We first tested HPCG and HPGMG on dual-socket CPU-based nodes and single-socket GPU-enabled nodes separately. After analyzing numerous configurations for the optimal performance, we moved on multiple program, multiple data (MPMD) runs to evaluate the performance on CPU-based XE6 nodes and GPU-enabled XK7 nodes together. ...

show abstract

HPC Benchmarking: Problem Size Matters

Cited by 9 publications

References 11 publications

HPC Benchmarking: Scaling Right and Looking Beyond the Average

HPC Benchmarking: Scaling Right and Looking Beyond the Average

Composite Measures of Supercomputer Technology

HPCG and HPGMG benchmark tests on multiple program, multiple data (MPMD) mode on Blue Waters—A Cray XE6/XK7 hybrid system

Contact Info

Product

Resources

About