Evaluating high‐performance computers

Abstract

SUMMARYComparisons of high-performance computers based on their peak floating point performance are common but seldom useful when comparing performance on real workloads. Factors that influence sustained performance extend beyond a system's floating-point units, and real applications exercise machines in complex and diverse ways. Even when it is possible to compare systems based on their performance, other considerations affect which machine is best for a given organization. These include the cost, the facilities requirements (power, floorspace, etc.), the programming model, the existing code base, and so on. This paper describes some of the important measures for evaluating high-performance computers. We present data for many of these metrics based on our experience at Lawrence Livermore National Laboratory (LLNL), and we compare them with published information on the Earth Simulator. We argue that evaluating systems involves far more than comparing benchmarks and acquisition costs. We show that evaluating systems often involves complex choices among a variety of factors that influence the value of a supercomputer to an organization, and that the high-end computing community should view cost/performance comparisons of different architectures with skepticism. Published in 2005 by John Wiley & Sons, Ltd.

Namibia's first high performance computer

“…Dongarra et al, 2003). Stating FLOPS is rarely useful for comparing real world performance (Vetter et al, 2005), however, it does help to give an overview of a cluster's possible capabilities and is a quick way to quantify these capabilities. Additionally, the wide use of the test allows for easy comparability.…”

BlueGene/L applications: Parallelism On a Massive Scale

“…This full installation of BG/L occupies only 2,500 square feet and has a total peak power consumption of only 1.2 MW. In comparison, the previous number one machine on the Top500 list, the Earth Simulator, occupies 34,000 square feet and has a peak power consumption of 10 MW (Vetter, de Supinski, May, Kissel, and Vaidya 2005), but achieves only 12.8% of BG/L's Linpack rate (University of Mannheim, University of Tennessee, and NERSC/LBNL ).…”