Generalized utility metrics for supercomputers

Strohmaier, Erich

doi:10.1007/s00450-009-0094-4

Cited by 3 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This power data was also added to the TOP500 list in June 2008. The performance-power ratio is criticized, because it is an intensive metric and thus cannot be used to rank supercomputers by size; however, the ratio is useful for ranking technologies [36].…”

Section: Energy Benchmarksmentioning

confidence: 99%

“…This can be seen from a figure of merit for mobile devices [19] to the SWaP metric for single servers [22] to a parameterized utility metric for supercomputers [36]. The construction principle is to multiply all desired quantities (such as performance) and divide them by undesired quantities (such as power and size).…”

Section: Composite Metricsmentioning

confidence: 99%

“…It is commonly measured in terms of a calculation rate such as floating point operations per second (FLOPS) or instructions per second. In fact, this type of metric is conveniently used to define a supercomputer [36]. For example, the TOP500 project [39] ranks computers by how quickly each can solve the LINPACK benchmark [6]; the first 500 are called "supercomputers".…”

Section: Introductionmentioning

confidence: 99%

“…For example, the Green500 project [13] reranks TOP500 supercomputers by LINPACK performance per watt (or equivalently, algorithmic steps per joule), referred to as FLOPS/W. However, this "miles per gallon" type of metric is criticized as being inappropriate for ranking supercomputers, due to its inability to track machines by size which may or may not reflect the total capability [36].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Towards efficient supercomputing

Hsu

Kuehn

Poole

2012

Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering

View full text Add to dashboard Cite

Efficiency in supercomputing has traditionally focused on execution time. In early 2000's, the concept of total cost of ownership was re-introduced, with the introduction of efficiency measure to include aspects such as energy and space. Yet the supercomputing community has never agreed upon a metric that can cover these aspects completely and also provide a fair basis for comparison. This paper examines the metrics that have been proposed in the past decade, and proposes a vector-valued metric for efficient supercomputing. Using this metric, the paper presents a study of where the supercomputing industry has been and where it stands today with respect to efficient supercomputing.

show abstract

Section: Energy Benchmarksmentioning

confidence: 99%

Section: Composite Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards efficient supercomputing

Hsu

Kuehn

Poole

2012

Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering

View full text Add to dashboard Cite

show abstract

“…Erich Strohmaier has pointed out that we must choose extensive metrics such as HPL that are able to rank systems in a manner that tracks growth of capability even if the metric does not reflect the rate of growth in the metics that are of most value to the user community [7]. Attempts to supplant HPL rankings have tended to fail because most common formulations of extensive metrics tend to correlate (albeit imprecisely) to the overall size of the machine.…”

Section: Historic Challenges To Creating a New Hpc Performance Metricmentioning

confidence: 99%

HPGMG 1.0: A Benchmark for Ranking High Performance Computing Systems

Adams¹,

Brown²,

Shalf³

et al. 2014

Self Cite

View full text Add to dashboard Cite

This document provides an overview of the benchmark -HPGMG -for ranking large scale general purpose computers for use on the Top500 list [8]. We provide a rationale for the need for a replacement for the current metric HPL, some background of the Top500 list and the challenges of developing such a metric; we discuss our design philosophy and methodology, and an overview of the specification of the benchmark. The primary documentation with maintained details on the specification can be found at hpgmg.org and the Wiki and benchmark code itself can be found in the repository https://bitbucket.org/hpgmg/hpgmg. SummaryThe High Performance Linpack (HPL) benchmark ) has been a successful metric for ranking high performance computing systems. HPL became a broadly accepted representative for application performance when it came to prominence in the 1990s, but over time has become less reflective of the system performance metrics that matter to contemporary science and engineering applications as lower complexity algorithms have been developed and required for extreme scale computing. We define a metric for ranking the worlds largest general purpose computers that maintains many of HPL's desirable qualities: a direct, non-iterative, solver (although only asymptotically exact) for systems of linear algebraic equations with a metric of equations solved per second (with a mapping to flops/s). We define a high performance geometric multigrid (HPGMG) benchmark that provides a more balanced exercise of machine capabilities, relative to application of interest in scientific computing, to provide a more accurate proxy for modern application requirements. HPGMG is composed of computations and data access patterns more commonly found in contemporary applications. Using HPGMG, we aim to create a benchmark for ranking systems that will promote system design improvements that are better aligned to real scientific application performance. IntroductionThe High Performance Linpack () benchmark is the most widely recognized and discussed metric for ranking high performance computing systems. When HPL gained prominence as a performance metric in the early 1990s there was a strong correlation between its predictions of system rankings and the ranking that full-scale applications would realize. Computer system vendors pursued designs that would increase HPL performance, which would in turn improve overall application performance.HPL rankings of computer systems are no longer so strongly correlated to real application performance, especially for the broad set of HPC applications that analyze differential equations, which tend to have much stronger needs for high bandwidth and low latency, and tend to access data using irregular patterns. In fact, we have reached a point where designing a system for good HPL performance can actually lead to design choices that are wrong for the real application mix, or add unnecessary components or complexity to the system. Despite that, due to its long accumulated history, the Top500 list continues to...

show abstract

The Right Metric for Efficient Supercomputing: A Ten-Year Retrospective

Hsu

Feng

2016

2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

View full text Add to dashboard Cite

Generalized utility metrics for supercomputers

Cited by 3 publications

References 8 publications

Towards efficient supercomputing

Towards efficient supercomputing

HPGMG 1.0: A Benchmark for Ranking High Performance Computing Systems

The Right Metric for Efficient Supercomputing: A Ten-Year Retrospective

Contact Info

Product

Resources

About