A global communication optimization technique based on data-flow analysis and linear algebra

Kandemir, Mahmut; Banerjee, P.; Choudhary, Alok; Ramanujam, J.; Shenoy, Niraj

doi:10.1145/330643.330647

Cited by 35 publications

(12 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Communication optimizations of parallel programs have been studied extensively in the context of data parallel languages [32,14,15,33,16,3]. Although some of the optimizations are similar to our work, the analysis problems are quite different: data parallel languages have a serial semantics without race conditions or memory consistency models, but they have an added burden of managing the mapping of fine-grained parallelism onto coarse-grained architectures.…”

Section: Related Workmentioning

confidence: 83%

Communication optimizations for fine-grained UPC applications

Chen

Iancu

Yelick

2005

14th International Conference on Parallel Architectures and Compilation Techniques (PACT'05)

View full text Add to dashboard Cite

Global address space languages like UPC exhibit high performance and portability on a broad class of shared and distributed memory parallel architectures. The most scalable applications use bulk memory copies rather than individual reads and writes to the shared space, but finergrained sharing can be useful for scenarios such as dynamic load balancing, event signaling, and distributed hash tables. In this paper we present three optimization techniques for global address space programs with fine-grained communication: redundancy elimination, use of split-phase communication, and communication coalescing. Parallel UPC programs are analyzed using static single assignment form and a dataflow graph, which are extended to handle the various shared and private pointer types that are available in UPC. The optimizations also take advantage of UPC's relaxed memory consistency model, which reduces the need for cross thread analysis. We demonstrate the effectiveness of the analysis and optimizations using several benchmarks, which were chosen to reflect the kinds of finegrained, communication-intensive phases that exist in some larger applications. The optimizations show speedups of up to 70% on three parallel systems, which represent three different types of cluster network technologies.

show abstract

Section: Related Workmentioning

confidence: 83%

Communication optimizations for fine-grained UPC applications

Chen

Iancu

Yelick

2005

14th International Conference on Parallel Architectures and Compilation Techniques (PACT'05)

View full text Add to dashboard Cite

show abstract

“…These techniques complement the communication optimizations introduced in this paper which are focused on reducing the number of bytes communicated and the number of remote tasks created in parallel objectoriented programs. There have been some compiler optimization frameworks that applied the techniques described above for Fortran D to array-based data-parallel programs written in languages such as HPF [11], [18], [8].…”

Section: Related Workmentioning

confidence: 99%

Communication Optimizations for Distributed-Memory X10 Programs

Barik

Zhao

Grove³

et al. 2011

2011 IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

Abstract-X10 is a new object-oriented PGAS (Partitioned Global Address Space) programming language with support for distributed asynchronous dynamic parallelism that goes beyond past SPMD message-passing models such as MPI and SPMD PGAS models such as UPC and Co-Array Fortran. The concurrency constructs in X10 make it possible to express complex computation and communication structures with higher productivity than other distributed-memory programming models. However, this productivity often comes at the cost of high performance overhead when the language is used in its full generality. This paper introduces high-level compiler optimizations and transformations to reduce communication and synchronization overheads in distributed-memory implementations of X10 programs. Specifically, we focus on locality optimizations such as scalar replacement and task localization, combined with supporting transformations such as loop distribution, scalar expansion, loop tiling, and loop splitting. We have completed a prototype implementation of these high-level optimizations, and performed a performance evaluation that shows significant improvements in performance, scalability, communication volume and number of tasks. We evaluated the communication optimizations on three platforms: a 128-node BlueGene/P cluster, a 32-node Nehalem cluster, and a 16-node Power7 cluster. On the BlueGene/P cluster, we observed a maximum performance improvement of 31.46× relative to the unoptimized case (for the MolDyn benchmark). On the Nehalem cluster, we observed a maximum performance improvement of 3.01× (for the NQueens benchmark) and on the Power7 cluster, we observed a maximum performance improvement of 2.73× (for the MolDyn benchmark). In addition, there was no case in which the optimized code was slower than the unoptimized case. We also believe that the optimizations presented in this paper will be necessary for any high-productivity PGAS language based on modern object-oriented principles, that is designed for execution on future Extreme Scale systems that place a high premium on locality improvement for performance and energy efficiency.

show abstract

“…So we only focus on minimizing synchronization and communication among the grid neighbors. Paper [18] presents a global communication optimization technique based on data-flow analysis and linear algebra. It aims at redundency elimination while our work also aims at combine non-redundent synchronizations in addition to eliminate redundent synchronizations.…”

Section: Related Workmentioning

confidence: 99%

Auto-CFD-NOW: A pre-compiler for effectively parallelizing CFD applications on networks of workstations

et al. 2006

View full text Add to dashboard Cite

Computational Fluid Dynamics (CFD) applications are highly demanding for parallel computing. Many such applications have been shifted from expensive MPP boxes to cost-effective Networks of Workstations (NOW). Auto-CFD-NOW is a pre-compiler that transforms Fortran CFD sequential programs to efficient message-passing parallel programs running on NOW. Our work makes the following three unique contributions. First, this precompiler is highly automatic, requiring a minimum number of user directives for parallelization. Second, we have applied a dependency analysis technique for the CFD applications, called analysis after partitioning. We propose a mirror-image decomposition technique to parallelize self-dependent field loops that are hard to parallelize by existing methods. Finally, traditional optimizations of communication focus on eliminating redundant synchronizations. We have developed an optimization scheme which combines all the non-redundant synchronizations in CFD programs to further reduce the communication overhead. The Auto-CFD-NOW has been implemented on networks of workstations and has been successfully used for automatically parallelizing structured CFD application programs. Our experiments show its effectiveness and scalability for parallelizing large CFD applications.

show abstract

A global communication optimization technique based on data-flow analysis and linear algebra

Cited by 35 publications

References 47 publications

Communication optimizations for fine-grained UPC applications

Communication optimizations for fine-grained UPC applications

Communication Optimizations for Distributed-Memory X10 Programs

Auto-CFD-NOW: A pre-compiler for effectively parallelizing CFD applications on networks of workstations

Contact Info

Product

Resources

About