Performance of Synchronized Iterative Processes in Multiprocessor Systems

Dubois, Michel; Briggs, Fayé A.

doi:10.1109/tse.1982.235576

Cited by 48 publications

(20 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If the subset consists of all the processors than the synchronization is a barrier synchroniza-~()(j C. S. CHANG AND R. NELSON tion. A performance model of an iterative algorithm executed on different architectures using barrier synchronization can be found in Dubois and Briggs [1982] and of partial synchronization in Brochard et al [1989]. Barrier Our interest in this paper is to determine how the type of synchronization affects the average utilization of the processors (commonly called the efficiency of the system [Brochard 1989;Engle et al 19891 In Section 2 of this paper, we present our models for the computation and architecture of the system we study.…”

Section: Introductionmentioning

confidence: 99%

Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

Chang

Nelson

1995

J. ACM

View full text Add to dashboard Cite

In this paper, we derive bounds on the speedup and efficiency of applications that schedule tasks on a set of parallel processors. We assume that the application runs an algorithm that consists of N iterations and before starting its [ + 1st iteration, a processor must wait for data (i.e., synchronize) calculated in the ith iteration by a subset of the other processors of the system. Processing times and interconnections between iterations are modeled by random variables with possibly deterministic distributions. Scientific applications consisting of iterations of recursive equations are examples of applications that can be modeled within this formulation.We consider the efficiency of such applications and show that, although efficiency decreases with an increase in the number of processors, it has a nonzero limit when the number of processors increases to infinity. We obtain a lower bound for the efficiency by solving an equation that depends on the distribution of task service times and the expected number of tasks needed to be synchronized. We also show that the lower bound is approached if the topology of the processor graph is "spread-out," a notion we define in the paper.

show abstract

Section: Introductionmentioning

confidence: 99%

Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

Chang

Nelson

1995

J. ACM

View full text Add to dashboard Cite

show abstract

“…For a small s under the normal distribution, all involved processors in the system generate mutual exclusion requests in a short period of time, likely to render the memory module containing the lock a hot spot. The request patterns generated by processors are affected by several factors, such as random delays and nondeterministic processing requirements [12], memory contention [13], the program's internal structure and control dependence graph [6], and so on.…”

Section: Methodsmentioning

confidence: 99%

A circular list based mutual exclusion scheme for large shared-memory multiprocessors

Tzeng²

1997

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Mutual exclusion in shared-memory multiprocessors is realized by employing a lock to determine the processor among those which compete for the critical section. Accesses to such a mutual exclusion lock may create heavy synchronization traffic and/or serious contention over the network, thereby degrading system performance considerably. In this paper, we introduce an efficient scheme which keeps synchronization traffic low and avoids serious hot-spot contention. This is made possible by constructing a circular list of the processors waiting for the critical section and by dispersing accesses to the lock. Extensive simulation of the proposed approach was conducted and the lower bound on the elapsed time was derived. Our simulation results demonstrate that the proposed scheme indeed achieves better performance than prior techniques, with its elapsed time close to the lower bound for the whole range of simulated system sizes, thus promising good scalability for large systems.

show abstract

“…Both of these have been cited as arguments for assuming nondeterministic task times [Kruskal and Weiss 1985;Dubois and Briggs 1982].…”

Section: Motivation For a Deterministic Modelmentioning

confidence: 99%

“…In contrast, previous analytic models for more detailed analysis of parallel program execution time [Dubois and Briggs 1982;Mohan 1984;Kruskal and Weiss 1985;Thomasian and Bay 1986;Kapelnikov et al 1989;Ammar et al 1990;Mak and Lundstrom 1990;Vrsalovic et al 1988;Cvetanovic 1987;Tsuei and Vernon 1990;Harzallah and Sevcik 1995;Xu et al 1996;Jonkers et al 1995] have been less successful, and are also less well understood. In practice, these models either require a specialized model derivation for each new parallel program, or, as explained in Section 6, they are restricted to particular program synchronization structures, task scheduling algorithms, or task execution time distributions that can be analyzed.…”

Section: Introductionmentioning

confidence: 95%

Parallel program performance prediction using deterministic task graph analysis

Adve

Vernon

2004

ACM Trans. Comput. Syst.

View full text Add to dashboard Cite

In this article, we consider analytical techniques for predicting detailed performance characteristics of a single shared memory parallel program for a particular input. Analytical models for parallel programs have been successful at providing simple qualitative insights and bounds on program scalability, but have been less successful in practice for providing detailed insights and metrics for program performance (leaving these to measurement or simulation). We develop a conceptually simple modeling technique called deterministic task graph analysis that provides detailed performance prediction for shared-memory programs with arbitrary task graphs, a wide variety of task scheduling policies, and significant communication and resource contention. Unlike many previous models that are stochastic models, our model assumes deterministic task execution times (while retaining the use of stochastic models for communication and resource contention). This assumption is supported by a previous study of the influence of nondeterministic delays in parallel programs. We evaluate our model in three ways. First, an experimental evaluation shows that our analysis technique is accurate and efficient for a variety of shared-memory programs, including programs with large and/or complex task graphs, sophisticated task scheduling, highly nonuniform task times, and significant communication and resource contention. The results also show that the deterministic assumption is crucial to permit accurate and yet efficient analysis of these programs. Second, we use three example programs to illustrate the predictive capabilities of the model. In two cases, broad insights and detailed metrics from the model are used to suggest improvements in load-balancing and the model quickly and accurately predicts the impact of these changes. In the third case, the model provides novel insights into the impact of program design changes that improve communication locality as well as load-balancing, via new (but general-purpose) metrics. Finally, we present results from a comparison of our model and representative stochastic models, and use these to characterize the conditions under which a deterministic model or stochastic models would be appropriate.

show abstract

Performance of Synchronized Iterative Processes in Multiprocessor Systems

Cited by 48 publications

References 11 publications

Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

Bounds on the speedup and efficiency of partial synchronization in parallel processing systems

A circular list based mutual exclusion scheme for large shared-memory multiprocessors

Parallel program performance prediction using deterministic task graph analysis

Contact Info

Product

Resources

About