Barrier Elimination Based on Access Dependency Analysis for OpenMP

Yonezawa, Naoki; Wada, Koichi; Aida, Takashi

doi:10.1007/11946441_36

Cited by 9 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many previous studies have optimized parallel loops to reduce task spawning and synchronization overheads [Heinz and Philippsen 1993;Tseng 1995;Yonezawa et al 2006]. Compared to these works that optimize only parallel loops, we have built a general compiler framework that focuses on eliminating arbitrary redundant finish barrier operations by applying sophisticated analysis and transformations.…”

Section: Barrier Synchronization and Task Creation Optimizationmentioning

confidence: 99%

A Transformation Framework for Optimizing Task-Parallel Programs

Nandivada

Shirako

Zhao

et al. 2013

ACM Trans. Program. Lang. Syst.

View full text Add to dashboard Cite

Task parallelism has increasingly become a trend with programming models such as OpenMP 3.0, Cilk, Java Concurrency, X10, Chapel and Habanero-Java (HJ) to address the requirements of multicore programmers. While task parallelism increases productivity by allowing the programmer to express multiple levels of parallelism, it can also lead to performance degradation due to increased overheads. In this article, we introduce a transformation framework for optimizing task-parallel programs with a focus on task creation and task termination operations. These operations can appear explicitly in constructs such as async, finish in X10 and HJ, task, taskwait in OpenMP 3.0, and spawn, sync in Cilk, or implicitly in composite code statements such as foreach and ateach loops in X10, forall and foreach loops in HJ, and parallel loop in OpenMP.Our framework includes a definition of data dependence in task-parallel programs, a happens-before analysis algorithm, and a range of program transformations for optimizing task parallelism. Broadly, our transformations cover three different but interrelated optimizations: (1) finish-elimination, (2) forall-coarsening, and (3) loop-chunking. Finish-elimination removes redundant task termination operations, forall-coarsening replaces expensive task creation and termination operations with more efficient synchronization operations, and loop-chunking extracts useful parallelism from ideal parallelism. All three optimizations are specified in an iterative transformation framework that applies a sequence of relevant transformations until a fixed point is reached. Further, we discuss the impact of exception semantics on the specified transformations, and extend them to handle task-parallel programs with precise exception semantics. Experimental results were obtained for a collection of task-parallel benchmarks on three multicore platforms: a dual-socket 128-thread (16-core) Niagara T2 system, a quad-socket 16-core Intel Xeon SMP, and a quad-socket 32-core Power7 SMP. We have observed that the proposed optimizations interact with each other in a synergistic way, and result in an overall geometric average performance improvement between 6.28× and 10.30×, measured across all three platforms for the benchmarks studied.This article is an extended and reorganized version of the following two previous articles from conference proceedings [Shirako et al. 2009;

show abstract

Section: Barrier Synchronization and Task Creation Optimizationmentioning

confidence: 99%

A Transformation Framework for Optimizing Task-Parallel Programs

Nandivada

Shirako

Zhao

et al. 2013

ACM Trans. Program. Lang. Syst.

View full text Add to dashboard Cite

show abstract

“…In the former study [7,8], the effect of the barrier eliminating algorithm is investigated experimentally. Figure 1 shows the ratio of improvement which is computed as 100 × (1 − T a /T b ), where T b is the execution time of Jacobi method before eliminating barriers and T a is the one after eliminating barriers.…”

Section: A Barrier Eliminating Algorithmmentioning

confidence: 99%

“…Figure 2 shows the results for Gauss method. The size of matrix is varied from 2,048 to 8,192. These applications are executed on a PC cluster which consists of Gigabit Ethernet and 16 PCs, each of which has Intel Xeon 2.8 GHz and 1 GB memory.…”

Section: A Barrier Eliminating Algorithmmentioning

confidence: 99%

“…However, barrier synchronization causes the processors' idle time to increase. To reduce the overhead of barrier synchronization, several methods [1,4,7,8] have been proposed. Tseng [4] proposed an algorithm to eliminate barrier synchronizations which appear SPMD (Single Program Multiple Data) program for shared memory multiprocessors and evaluate it on Stanford DASH [2].…”

mentioning

confidence: 99%

“…This method assumes the existence of a shared memory which DASH provides. Dwarkadas et al[1] proposed a method for their compiler to replace a barrier synchronization with push function in which producer sends updated data to consumers.In our method [7,8] which targets on the distributed shared memory environment realized on a PC cluster, the compiler analyzes the dependency between the data producers and the data consumers. Then, the compiler replaces the barrier synchronization with message passing code which sends the data to consumer's side.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Probabilistic Analysis of Load-Imbalanced Parallel Applications With Partially Eliminated Barriers

Yonezawa

Katou

Kino

et al. 2015

JORSJ

Self Cite

View full text Add to dashboard Cite

In order to reduce the overhead of barrier synchronization, we have proposed an algorithm which eliminates barrier synchronizations and evaluated its validity experimentally in our previous study. As a result, we have found that the algorithm is more effective to the load-imbalanced program than load-balanced program. However, the degree of the load balance has not been discussed quantitatively. In this paper, we model the behavior of parallel programs. In our model, the execution time of a phase contained in a parallel program is represented as a random variable. To investigate how the degree of the load balance influences the performance of our algorithm, we varied the coefficient of variation (CV) of probability distribution which the random variable follows. Using the model, we evaluated the execution time of parallel programs which have four typical dependency patterns. Based on results, we found that theoretical results are consistent with experimental ones.Keywords: Information technologies, barrier elimination, probabilistic analysis, time reduction IntroductionSince barrier synchronization is a simple means to guarantee the order of data producing and data consuming, it is often used in parallel programs. However, barrier synchronization causes the processors' idle time to increase. To reduce the overhead of barrier synchronization, several methods [1,4,7,8] have been proposed. Tseng [4] proposed an algorithm to eliminate barrier synchronizations which appear SPMD (Single Program Multiple Data) program for shared memory multiprocessors and evaluate it on Stanford DASH [2]. He also proposed another algorithm to replace a barrier synchronization with a counter-based synchronization. The latter algorithm, in which consumers wait until a producer updates a variable, realizes synchronization among processors with smaller overhead than barrier synchronization. This method assumes the existence of a shared memory which DASH provides. Dwarkadas et al.[1] proposed a method for their compiler to replace a barrier synchronization with push function in which producer sends updated data to consumers.In our method [7,8] which targets on the distributed shared memory environment realized on a PC cluster, the compiler analyzes the dependency between the data producers and the data consumers. Then, the compiler replaces the barrier synchronization with message passing code which sends the data to consumer's side. Through evaluation, we have found that the algorithm is more effective to the load-imbalanced program than load-balanced program. However, the degree of the load balance has not been discussed quantitatively.In this paper, in order to analyze the effect of the method theoretically, we propose a probabilistic model to describe the degree of the load balance and evaluate the effectiveness

show abstract

A Synchronization Optimization Technique for OpenMP

Deng

Lin

2021

2021 IEEE 13th International Conference on Computer Research and Development (ICCRD)

View full text Add to dashboard Cite

Barrier Elimination Based on Access Dependency Analysis for OpenMP

Cited by 9 publications

References 9 publications

A Transformation Framework for Optimizing Task-Parallel Programs

A Transformation Framework for Optimizing Task-Parallel Programs

Probabilistic Analysis of Load-Imbalanced Parallel Applications With Partially Eliminated Barriers

A Synchronization Optimization Technique for OpenMP

Contact Info

Product

Resources

About