Improving sparse data movement performance using multiple paths on the Blue Gene/Q supercomputer

Abstract

In situ analysis has been proposed as a promising solution to glean faster insights and reduce the amount of data to storage. A critical challenge here is that the reduced dataset is typically located on a subset of the nodes and needs to be written out to storage. Data coupling in multiphysics codes also exhibits a sparse data movement pattern wherein data movement occurs among a subset of nodes. We evaluate the performance of data movement for sparse data patterns on the IBM Blue Gene/Q supercomputing system "Mira" and identify performance bottlenecks. We propose a multipath data movement algorithm for sparse data patterns based on an adaptation of a maximum flow algorithm together with breadth-first search that fully exploits all the underlying data paths and I/O nodes to improve data movement. We demonstrate the efficacy of our solutions through a set of microbenchmarks and application benchmarks on Mira scaling up to 131,072 compute cores. The results show that our approach achieves up to 5X improvement in achievable throughput compared with the default mechanisms.

Textile Resistance Switching Memory for Fabric Electronics

“…Computer evolution has now diverged into two main areas: first, the pursuit of hyper‐performance, such as mainframe supercomputers, and second, the drive for “invisible” computing instruments with relatively low computing power. Currently, wearable computers and devices are representative examples of the latter.…”