The improvement of mechanisms for program representation of parallelism was. and still is, the central problem of parallel programming. In practice, this consists of two different problems:the development of methods for describing parallelism in programs modelling complex discrete systems and processes;the development of methods for the organization of computations on parallel multiprocessing computers.More specifically, the problem lies in the development of parallel programming languages containing facilities for the description of task parallelism and/or the definition of computational parallelism. Both problems have many traits in common from the point of view of the basic principles underlying the mechanism of parallel structure organization. Basically, the differences between them can be reduced to the differences between high-and low-level languages.
Parallelism in programs and computationsThe discussion of problems associated with parallelism in programming requires a more precise definition of this concept, the separation of its abstract and concrete forms at various stages of programming, and at various levels of the program hierarchy.Difficulties arise from the very beginning : how to define the concepts of parallel computation (processes, statements, operations, etc.). The definitions introducing real or conditional time and leading to ideas such as "concurrent processes" or "processes that can be concurrent", lead to difficulties and ambiguities unless one considers trivial examples (e.g. a pair of processes which do not interact at all). The fact that formal theoretical definitions of parallelism based on time relations are complicated reflects these difficulties. There is another, wider possibility for defining parallelism : fragments or processes are referred to as parallel, if they are (or can be) implemented by different separate physical or virtual units. One can fully avoid the necessity of defining the parallelism relation and, instead, define the dependence relation of fragments or processes. This approach complements the above ones, and it can be characterized as "parallelism presumption": all the program fragments and actions are regarded as initially independent, i.e. parallel;