Implications of application usage characteristics for collective communication offload

Brightwell, Ron; Goudy, Sue P.; Rodrigues, Arun F.; Underwood, Keith D.

doi:10.1504/ijhpcn.2006.010633

Cited by 23 publications

(15 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance benefits of overlapping communication with computation have been well studied [6]. Some have further explored how these techniques can be applied to collectives and their benefits in applications that are written in two-sided communication models [7,8].…”

Section: Nonblocking Collectivesmentioning

confidence: 99%

Tuning collective communication for Partitioned Global Address Space programming models

et al. 2011

View full text Add to dashboard Cite

Partitioned Global Address Space (PGAS) languages offer programmers the convenience of a shared memory programming style combined with locality control necessary to run on large-scale distributed memory systems. Even within a PGAS language programmers often need to perform global communication operations such as broadcasts or reductions, which are best performed as collective operations in which a group of threads work together to perform the operation. In this paper we consider the problem of implementing collective communication within PGAS languages and explore some of the design trade-offs in both the interface and implementation. In particular, PGAS collectives have semantic issues that are different than in send-receive style message passing programs, and different implementation approaches that take advantage of the one-sided communication style in these languages. We present an implementation framework for PGAS collectives as part of the GASNet communication layer, which supports shared memory, distributed memory and hybrids. The framework supports a broad set of algorithms for each collective, over which the implementation may be automatically tuned. Finally, we demonstrate the benefit of optimized GASNet collectives using application benchmarks written in UPC, and demonstrate that the GASNet collectives can deliver scalable performance on a variety of state-of-the-art parallel machines including a Cray XT4, a IBM BlueGene/P, and a Sun Constellation system with InfiniBand interconnect.

show abstract

Section: Nonblocking Collectivesmentioning

confidence: 99%

Tuning collective communication for Partitioned Global Address Space programming models

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Though further testing is needed, current performance analysis results suggest this decrease in application runtime is due to unexpected messages. A CTH run is known to contain a large number of unexpected messages [3]. With parallel's sender side synchronization protocol, many of these unexpected messages will now arrive after the receive has been posted and therefore slightly increase CTH's performance.…”

Section: Lammpsmentioning

confidence: 99%

rMPI : increasing fault resiliency in a message-passing environment.

Stearley¹,

Laros²,

Ferreira³

et al. 2011

View full text Add to dashboard Cite

As High-End Computing machines continue to grow in size, issues such as fault tolerance and reliability limit application scalability. Current techniques to ensure progress across faults, like checkpointrestart, are unsuitable at these scale due to excessive overheads predicted to more than double an applications time to solution. Redundant computation, long used in distributed and mission critical systems, has been suggested as an alternative to checkpoint-restart on its own. In this paper we describe the rMPI library which enables portable and transparent redundant computation for MPI applications. We detail the design of the library as well as two replica consistency protocols, outline the overheads of this library at scale on a number of real-world applications, and finally outline the significant increase in an applications time to solution at extreme scale as well as show the scenarios in which redundant computation makes sense.

show abstract

“…This has the advantage of using a commodity off-the-shelf interconnect (Gigabit Ethernet, in this case) in a commodity cluster. These ideas were further explored in [17]. Voltaire has recently announced support for collective communications inside of their InfiniBand switch; however, no peer-reviewed report is available yet to characterize the advantages.…”

Section: B Hardware Approachesmentioning

confidence: 99%

Impact of reconfigurable hardware on accelerating MPI_Reduce

Gao

Schmidt

Sass

2010

2010 International Conference on Field-Programmable Technology

View full text Add to dashboard Cite

This paper demonstrates the benefits and pit-falls of implementing the collective communication operation reduce in the reconfigurable resources of an FPGA device across a cluster of all-FPGA compute nodes. Specifically, the communication and computation semantics of the MPI_Reduce call from the de facto Message-Passing Interface have been implemented. Using a synthetic benchmark a cluster of 32 FPGA nodes with a 300 MHz PowerPC processor, custom high speed network, and reduce core is compared against a conventional commodity cluster with 3.2 GHz Xeon processors and Gigabit Ethernet. The design is customized to support performing many reduce operations on small datasets while minimizing the amount of on-chip resources used, which is an increasingly common demand from domain scientists. Speedups of ≈2× to ≈800× are reported over that of a commodity cluster for small datasets, which provides significant motivation to continue the investigation into supporting additional collective communication operations directly in hardware.

show abstract

Implications of application usage characteristics for collective communication offload

Cited by 23 publications

References 33 publications

Tuning collective communication for Partitioned Global Address Space programming models

Tuning collective communication for Partitioned Global Address Space programming models

rMPI : increasing fault resiliency in a message-passing environment.

Impact of reconfigurable hardware on accelerating MPI_Reduce

Contact Info

Product

Resources

About