A quantitative study of parallel scientific applications with explicit communication

Abstract

This paper studies the behavior of scientific applications running on distributed memory parallel computers. Our goal is to quantify the floating point, memory, I/O, and communication requirements of highly parallel scientific applications that perform explicit communication. In addition to quantit).ing these requirements for fixed problem sizes and numbers of processors, we develop analytical models for the effects of changing the problem size and the degree of parallelism for several of the applications.The contribution of our paper is that it provides quantitative data about real parallel scientific applications in a manner that is largely independent of the specific machine on which the application was run. Such data, which are clearly very valuable to an architect who is designing a new parallel computer, were not previously available. For example, the majority of research papers in interconnection networks have used simulated communication loads consisting of fixed-size messages. Our data, which show that using such simulated loads is unrealistic, can be used to generate more realistic communicalion loads.

Workload Modeling for Computer Systems Performance Evaluation

“…This extensive standard defines multiple types of point-to-point and collective communication primitives, thus enabling the characterization of communication patterns at a relatively high level of abstraction. Results of studies of MPI activity reveal a wide distribution of message sizes, rates, and blocks of computation between messages [707], as do studies based on earlier proprietary interfaces [160]. However, each individual application typically has a rather modal distribution of sizes (Figure 9.44).…”

“…Note the logarithmic Y scale of rates, which are normalized per gflop of computation on the entire machine. Data from [160].…”

Workload Modeling for Performance Evaluation

“…The question is what distributions to use. While there has been some work done on characterizing specific applications [20,74,65], there has been little if any work on characterizing the mix of application characteristics in a typical workload. A rather singular example is the Charisma project, in which a whole workload was measured [59].…”