Blue Gene/L programming and operating environment

Abstract

With up to 65,536 compute nodes and a peak performance of more than 360 teraflops, the Blue Genet/L (BG/L) supercomputer represents a new level of massively parallel systems. The system software stack for BG/L creates a programming and operating environment that harnesses the raw power of this architecture with great effectiveness. The design and implementation of this environment followed three major principles: simplicity, performance, and familiarity. By specializing the services provided by each component of the system architecture, we were able to keep each one simple and leverage the BG/L hardware features to deliver high performance to applications. We also implemented standard programming interfaces and programming languages that greatly simplified the job of porting applications to BG/L. The effectiveness of our approach has been demonstrated by the operational success of several prototype and production machines, which have already been scaled to 16,384 nodes.

The impact of network noise at large-scale communication performance

“…The generated Fat Tree topologies are shown in Table 1. Figure 9 shows Layout # Endpoints # Switches xgft (2,12,6) 144 18 xgft (2,24,12) 288 36 xgft (3,12,8,12,4) 1152 240 xgft (3,12,16,12,8) 2304 480 xgft (3,12,24,12,12) 3456 720 xgft (4,12,12,6,12,12,3) 10368 3024 xgft (4,12,12,12,12,12,6) 20736 6048 …”

“…Host-side delays are caused by such things as hardware interrupts, translation lookaside buffer (TLB) misses, context switches or cache misses-all of which result from time-sharing the main CPU among different processes. Due to the serial nature and obvious source of such overheads, there are some obvious approaches to minimize their influence, such as the use of low-noise operating systems specialized for high performance computing (HPC) (such as Catamount [7] or Blue Gene Linux [8]). …”

A system level view of Petascale I/O on IBM Blue Gene/P

“…The Blue Gene/P system design [1] is very similar to Blue Gene/L [2][3][4]. Blue Gene compute nodes (CNs) run the user application on a lightweight compute node kernel (CNK), and do not have direct TCP/IP connection to an external network.…”

Measuring power consumption on IBM Blue Gene/P

“…Regarding energy efficiency, the key feature of Blue Gene is its judiciously chosen low-frequency, low-voltage design which results in both high-performance and highly energy-efficient supercomputers. Blue Gene/L [6][7][8] and Blue Gene/P [9] systems have consistently gained top MFlops/Watt rankings on the Green500 list, and an early prototype of the next generation Blue Gene/Q system has recently set a new record at 2.1 GFlops/Watt [2]. One important aspect of the familiarity design principle is that the well established MPI parallel programming paradigm on homogeneous nodes is main- Fig.…”