Global combine on mesh architectures with wormhole routing

Abstract

data elements per node. We also introduce a hybrid algorithm that is not asymptotically optimal, but in Several algorithms are discussed for implementing practice outperforms the others for wide ranges of n global combine (summation) oll distributed memory and p. In addition, we show that a different algocomputers using a two-dimensional mesh interconnect rithm, optimized for a hypercube, is often the fastest with wormhole routing. These include algorithms that method for meshes containing p = 2d nodes, if care is are asymptotically optimal for short vectors (O(log(p)) taken to order the communications to minimize net-/or p processing nodes) and /or long vectors (O(n) work contention. The state of the art for performing /or n data elements per node), as well as hybrid ai-the global combine on hypercubes is described in [5]. gorithms that are superior for intermediate n. Per-Earlier work on 2-D mesh combining is reported in [2]. formance models are developed that include the effects The global combine operation can be stated as folof link conflicts and other characteristics of the un-lows: Given p processing nodes, each of which owns a derlying communication system. The models are val-vector of data, xi, of length n, a global combine forms p-1 idated using experimental data from the Intel Touchy = (9i= 0 (xi), where $ is a commutative and associastone DELTA computer. Each of the combine algo-tive operator defined on the elements of the vectors. rithms is shown to be superior under some circum-In this paper we choose to have a copy of the resulting stances, y end up on every node.

C3: A Parallel Model for Coarse-Grained Machines

“…Our results also indicate that the performance of an implementation is influenced by the relationship among parameters of the parallel machine, as well as by the relationship of the parameters to the amount of data involved. This agrees with other research done on the implementation of communication operations [1,2,4,19].…”

Broadcasting on Meshes with Wormhole Routing

“…Ideas from our previous work on performing the global combine can be used to obtain an alternative tradeoff between the startup cost and the transfer cost [4]. We first present a simple algorithm for one-dimensional meshes, and then extend it to the two-dimensional case.…”

NIC-based reduction algorithms for large-scale clusters

“…Reduction collectives entail both communication (data transfer) and processing (data reduction operations), and therefore efficient implementations must consider the characteristics of the network, the processor, and the interactions between them. Over the years, many researchers have dedicated significant effort to derive optimal and scalable algorithms [1,2,3,4,5,8]. However, with respect to the underlying system characteristics, all of this work commonly assumed reduction processing must be performed by the host CPU.…”