Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

Chang, Cheng-Shang; Nelson, Randolph

doi:10.1145/200836.200872

Cited by 10 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also give a new proof of the constant bound for normally distributed tasks shown by Chang and Nelson [1]. Finally, we show that for certain power-law distributions, the time between tasks for a processor goes to infinity as the number of processors increases, even for neighbor synchronization, as long as the graph is strongly connected.…”

Section: Introductionmentioning

confidence: 71%

“…Furthermore, the expected time for a node along a path in the loop induced by this synchronization is less than or equal to the unconditional expectation of the task distribution, since processors that finish tasks earlier are included along more paths than those that finish later. Using these two facts, we can apply their supermartingale that sums along all possible paths and show that their bound applies here (for more details, see [1]. )…”

Section: The First-neighbor Modelmentioning

confidence: 99%

See 1 more Smart Citation

A performance analysis of local synchronization

Lipman

Stout

2006

Proceedings of the Eighteenth Annual ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

Synchronization is often necessary in parallel computing, but it can create delays whenever the receiving processor is idle, waiting for the information to arrive. This is especially true for barrier, or global, synchronization, in which every processor must synchronize with every other processor. Nonetheless, barriers are the only form of synchronization explicitly supplied in MPI and OpenMP.Many applications do not actually require global synchronization; local synchronization, in which a processor synchronizes only with those processors from which it has an incoming edge in some directed graph, is often adequate. However, the behavior of a system under local synchronization is more difficult to analyze, since processors do not all start tasks at the same time.In this paper, we show that if the synchronization graph is a directed cycle and the task times are geometrically distributed with p = 0.5, the time it takes for a processor to complete a task, including synchronization time, approaches an exact limit of 2 + √ 2 as the number of processors in the cycle approaches infinity. Under global synchronization, however, the time is unbounded, increasing logarithmically with the number of processors. Similar results also apply for p = 0.5.We give a new proof of the constant upper bounds that apply when tasks are normally distributed and the synchronization graph is any graph of bounded degree. We also prove that for some power-law distributions on the tasks, there is no constant upper bound as the number of processors increases, even for the directed cycle. Finally, we show that constant upper bounds apply for some cases of a different synchronization model in which a processor waits for only a subset of its neighbors.

show abstract

Section: Introductionmentioning

confidence: 71%

Section: The First-neighbor Modelmentioning

confidence: 99%

A performance analysis of local synchronization

Lipman

Stout

2006

Proceedings of the Eighteenth Annual ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

show abstract

“…Issuing tasks in parallel or gang scheduling is tantamount to a fork, and barrier synchronization to await the completion of all tasks is a join. Bounds on speedup and efficiency are obtained in Chang and Nelson [1995], when partial synchronization is required. The analysis is complicated, as some iterations start while others are still in progress.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Fork/Join and Related Queueing Systems

Thomasian¹

2014

ACM Comput. Surv.

View full text Add to dashboard Cite

Fork/join (F/J) requests arise in contexts such as parallel computing, query processing in parallel databases, and parallel disk access in RAID. F/J requests spawn K tasks that are sent to K parallel servers, and the completion of all K tasks marks the completion of an F/J request. The exact formula for the mean response time of K = 2-way F/J requests derived under Markovian assumptions (R F/J 2 ) served as the starting point for an approximate expression for R F/J K for 2 < K ≤ 32. When servers process independent requests in addition to F/J requests, the mean response time of F/J requests is better approximated by R max K , which is the maximum of the response times of tasks constituting F/J requests. R max K is easier to compute and serves as an upper bound to R F/J K . We discuss techniques to compute R max K and generally the maximum of K random variables denoting the processing times of the tasks of a parallel computation X max K . Graph models of computations such as Petri nets-a more general form of parallelism than F/J requests-are also discussed in this work. Jobs with precedence constraints may require multiple resources, which are represented by a queueing network model. We also discuss various queueing systems related to F/J queueing systems and outline their analysis. ACM Reference Format:Alexander Thomasian. 2014. Analysis of fork-join and related queueing systems.

show abstract

“…The efficiency by solving a recursive equation that depends on the distribution of task service times and the expected number of tasks needed to be synchronized, efficiency decreases with an increase in the number of processors [10]. Eliminating barrier synchronization [11] for compiler-parallelized codes on software distributed shared memory.…”

Section: Introductionmentioning

confidence: 99%

Distributed Computing Design Methods for Multicore Application Programming

Yu¹,

Li²,

Xie³

et al. 2012

Proceedings of the 2012 2nd International Conference on Computer and Information Applications (ICCIA 2012)

View full text Add to dashboard Cite

In order to solve the serial execution caused by multithreaded concurrent access to shared data and realize the dynamic load balance of tasks on shared memory symmetric multi-processor (multi-core) computing platform, new design methods are presented. By presenting multicore distributed locks, multicore shared data localization, multicore distributed queue, the new design methods can greatly decrease the number of accessing the shared data and realize the dynamic load balance of tasks. For illustration, design scheme of multicore task manager of server software are given by using new design methods. Results shows the new design methods reduce the number of access shared resources, partially resolve the serial execution of cooperative threads and realize the dynamic task balance of server software, which validate the superiority of this approach.

show abstract

Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

Cited by 10 publications

References 23 publications

A performance analysis of local synchronization

A performance analysis of local synchronization

Analysis of Fork/Join and Related Queueing Systems

Distributed Computing Design Methods for Multicore Application Programming

Contact Info

Product

Resources

About