Optimal synchronization frequency for dynamic pipelined computations on heterogeneous systems

Ciorba, Florina M.; Riakiotakis, I.; Andronikos, Theodore; Chronopoulos, Anthony Theodore; Παπακωνσταντίνου, Γ.

doi:10.1109/clustr.2007.4629257

Cited by 4 publications

(11 citation statements)

References 19 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applying self-scheduling algorithms to nested loops with data dependencies leads to a pipelined execution because of the data dependencies [34,42]. In fine-grain pipelining, the pipeline parallelism can be increased at the cost of increased communication overhead.…”

Section: Pipeline Scheduling Of Loops With Data Dependenciesmentioning

confidence: 99%

“…The first case .A D N / is a novel contribution to the work in [34]. This is because more work is assigned to faster processors.…”

Section: The Theoretical Parallel Execution Modelmentioning

confidence: 99%

“…The case of limited and unlimited number of processors in a heterogeneous system is investigated, and the proposed model is applied for the CSS, FSS, GSS, and TSS self‐scheduling algorithms on a heterogeneous distributed system. Previous efforts to address the same problem were presented . The work in this paper improves on both as follows.…”

Section: Introductionmentioning

confidence: 98%

“…The current communication model parameters are application‐independent and are estimated with the help of the mpptest from the perftest suite . In contrast to the communication model used in , the current communication model takes into account the effects of message size and network contention on the communication cost, as the mpptest benchmark was specifically designed for this purpose at Argonne National Laboratory. The mpptest benchmark also exposes scalability issues.…”

Section: Introductionmentioning

confidence: 99%

“…The work in this paper improves on both as follows. The communication model used herein is more sophisticated than the one in [34]. The current communication model parameters are application-independent and are estimated with the help of the mpptest from the perftest suite [36].…”

mentioning

confidence: 99%

See 4 more Smart Citations

Towards the optimal synchronization granularity for dynamic scheduling of pipelined computations on heterogeneous computing systems

Riakiotakis

Ciorba

Andronikos

et al. 2012

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Loops are the richest source of parallelism in scientific applications. A large number of loop scheduling schemes have therefore been devised for loops with and without data dependencies (modeled as dependence distance vectors) on heterogeneous clusters. The loops with data dependencies require synchronization via cross-node communication. Synchronization requires fine-tuning to overcome the communication overhead and to yield the best possible overall performance. In this paper, a theoretical model is presented to determine the granularity of synchronization that minimizes the parallel execution time of loops with data dependencies when these are parallelized on heterogeneous systems using dynamic self-scheduling algorithms. New formulas are proposed for estimating the total number of scheduling steps when a threshold for the minimum work assigned to a processor is assumed. The proposed model uses these formulas to determine the synchronization granularity that minimizes the estimated parallel execution time. The accuracy of the proposed model is verified and validated via extensive experiments on a heterogeneous computing system. The results show that the theoretically optimal synchronization granularity, as determined by the proposed model, is very close to the experimentally observed optimal synchronization granularity, with no deviation in the best case, and within 38.4% in the worst case.

show abstract

Section: Pipeline Scheduling Of Loops With Data Dependenciesmentioning

confidence: 99%

“…The first case .A D N / is a novel contribution to the work in [34]. This is because more work is assigned to faster processors.…”

Section: The Theoretical Parallel Execution Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%