An early performance evaluation of many integrated core architecture based SGI rackable computing system

Abstract

Intel recently introduced the Xeon Phi coprocessor based on the Many Integrated Core architecture featuring 60 cores with a peak performance of 1.0 Tflop/s. NASA has deployed a 128-node SGI Rackable system where each node has two Intel Xeon E2670 8-core Sandy Bridge processors along with two Xeon Phi 5110P coprocessors. We have conducted an early performance evaluation of the Xeon Phi. We used microbenchmarks to measure the latency and bandwidth of memory and interconnect, I/O rates, and the performance of OpenMP directives and MPI functions. We also used OpenMP and MPI versions of the NAS Parallel Benchmarks along with two production CFD applications to test four programming modes: offload, processor native, coprocessor native and symmetric (processor plus coprocessor). In this paper we present preliminary results based on our performance evaluation of various aspects of a Phi-based system.

Early Multi-node Performance Evaluation of a Knights Corner (KNC) Based NASA Supercomputer

“…It should be noted that performance of MPI functions in native MIC mode is 3 to 20 times worse than in native host mode as reported by Saini et al [13]. Poor scalability for BT and SP on MIC is because of load imbalance using the pure MPI paradigm.…”

“…Applications with significant amounts of MPI communication, especially collective communication, perform very poorly on MIC because the performance of MPI functions is 3 to 20 times slower for intra-MIC and 10 to 60 times slower for inter-MIC communication as compared to host [13]. To reduce MPI communication time, we performed optimization by packing and unpacking the MPI messages.…”

“…Many researchers investigating KNC performance have examined single nodes, where one or two Xeon processors on a single host are combined with one or two KNC coprocessors [7][8][9][10][11][12][13][14][15]. There are not, however, many publications on programming multiple nodes with MIC coprocessors.…”

“…Joo et al demonstrated a fully 'native' multi-node LQCD implementation running entirely on KNC nodes with minimum involvement of the host processor with strong scaling to 3.6 Tflop/s on 64 KNC [11]. Saini et al did performance evaluation of a single MIC using several low level benchmarks and two applications [13]. They measured STREAM bandwidth, load latency, load read and write bandwidth for L1/L2/L3 cache and main memory, MPI latency and bandwidth, and offload bandwidth between host and MIC0/MIC1.…”

“…one node (one host + MIC0 + MIC1). In the present paper we evaluate the performance of multiple nodes [13], optimize two large-scale production quality codes and implement a new technique to load balance for symmetric mode, which enhanced their performance significantly.…”