How to use up processors

Abstract

Most efforts to realize parallel systems presuppose three distinct principles the conjunction of which would seem to conflict with current and anticipated developments in architecture. Thus, most parallelization aims to decompose problems into (1) data independent units (2) each of which contributes to the final output i n the dataflow and which (3)minimizes synchronization. Both theoretical reflection and common practice suggest that decreasing the average data space of a collection of processes will increase occassions for synchronization. The paradigm of 'possible worlds computing' aims to explore a model in which parallelization is achieved by disregarding the first two principles in order to preserve the third. This paper describes the Tahiti Programming Language and its supporting run-time kernel Syinphora which are intended to explore the possible worlds paradigm.The vast majority of efforts towards palallelism presuppose two quite distinrt principles governing the decomposition of computations and the use of computing resoiirces. These are:1. Data Independelice. Prohlems slioultl he decomposed into computational units which manipulate data not shared with other units. E'cncy.A decomposition is good only when each independent parallel unit contrihutcs causally---in the dat.a flow-to the outcome of a coniput.ation.To some resea.rchers these assuinptions axe so compelling that they are never called into question o r even explicitly articulated. But. there is reason to believe that they are on a collision course with current developments in architecture. For they conflict with a third widely acknowledged principle, namely, 3. .4vo/d L~ynch~~onzzatzon. Computations should be orga-CH2908-2/90/0000/0515/$01.00 0 1990 IEEE I l l 11 nized so as to minimize blocking.The smaller the decomposition of a problem into units with small data. spaces, the more frequent communication between units must occur. Needless to say. a.rchitectures favored with many siniulta.neously ava.ilable resources seem to require just this pa.rallelization strategy. Ta.king a.dvantage of current and planned massively parallel architectures entails a confIonta.tion with at least t,nm of tlie three principles, namely, I and 3. The finer the granularity of a decomposit.ion the more likely units will conflict in their read/write access to data. and hence to require explicit synchronization.One solution to this problem is adopt a paradigm of parallel computing in which principle 3 is observed by relaxing 1 and 2. On this paradigm pa.rallel computations are organized into collections of processes, some of which are operat.ing on assumptions about t,he oiit,come of other processes, with which they are ertculzng sin, u / l a l~e o d 9 .On this paradigrn, efica.cy is violated because computations are orga.nized i n such a way that, some computational work is -strictly speaking -wasted. Data Independence is violated because distinct processes will he working on--what a.ppears to l)e--exact,ly the same d a h . Tliis paradigm we dub the 'possible worlds ...