Communication-Sensitive Static Dataflow for Parallel Message Passing Applications

Abstract

Message passing is a very popular style of parallel programming, used in a wide variety of applications and supported by many APIs, such as BSD sockets, MPI and PVM. Its importance has motivated significant amounts of research on optimization and debugging techniques for such applications. Although this work has produced impressive results, it has also failed to fulfill its full potential. The reason is that while prior work has focused on runtime techniques, there has been very little work on compiler analyses that understand the properties of parallel message passing applications and use this information to improve application performance and quality of debuggers.This paper presents a novel compiler analysis framework that extends dataflow to parallel message passing applications on arbitrary numbers of processes. It works on an extended controlflow graph that includes all possible inter-process interactions of any numbers of processes. This enables dataflow analyses built on top of this framework to incorporate information about the application's parallel behavior and communication topology. The parallel dataflow framework can be instantiated with a variety of specific dataflow analyses as well as abstractions that can tune the accuracy of communication topology detection against its cost.The proposed framework bridges the gap between prior work on parallel runtime systems and sequential dataflow analyses, enabling new transformations, runtime optimizations and bug detection tools that require knowledge of the application's communication topology. We instantiate this framework with two different symbolic analyses and show how these analyses can detect different types of communication patterns, which enables the use of dataflow analyses on a wide variety of real applications. I. IntroductionThe rise of cluster and multi-core computing has driven a revolution in computer hardware and software. As these machines are best utilized by parallel applications that are explicitly written to take advantage of multiple processing cores, their popularity is pushing the development of a wide variety of parallel applications. In particular, message passing applications, where processes use distributed memory and communicate via explicit send and receive operations, have become very popular. These applications are common on a wide variety of platforms and use popular APIs such as BSD sockets, MPI and PVM.The past few decades have seen significant amounts of work on message passing optimization and applications. The bulk of it has focused on runtime and techniques that either improve the underlying hardware (e.g. network processors [12]), the software infrastructure (e.g. self-tuning MPI [6]) or use run-time information to analyze and tune application performance [11]. In contrast, work on static techniques to analyze and optimize such applications has been mostly limited to sequential analyses that either improve the performance of the sequential portions of these applications or improve parallel performance by looking at th...

A Sound Reduction of Persistent-Sets for Deadlock Detection in MPI Applications

“…To gather the afore-said information, we would require an MPI specific control flow graph (CFG). Work in [1] presents pCFG which is a CFG for MPI programs. Our future work would therefore lie in modifying the pCFG work to handle non-deterministic MPI operations.…”

“…Informal testing approaches for MPI (e.g., based on schedule perturbation [18]) do not guarantee coverage, and are also highly redundant because they will, in practice, generate many equivalent schedules (e.g., permuting deterministic message match pairs). While static analyzers for MPI exist (e.g., [1]), they are known to be unsound (can generate too many false alarms) when used for bug-hunting, due to their overapproximation of possible message matches. Modelchecking based methods (e.g., MPI-SPIN [12]) can guarantee coverage, but on models of MPI programs; such models are very difficult to create, and become obsolete with each design change.…”

Static Approximation of MPI Communication Graphs for Optimized Process Placement

“…However, they have limitations that prevent their application to the problem described. Noted by multiple sources are the SPMD semantics of MPI [5,18,25]. The SPMD semantics are important as they largely define the methods that can be, and are, used to perform communication analysis.…”

“…Our analysis is best-e↵ort, prioritizing coverage over soundness; for instance we assume global variables are only modified by compiletime visible functions. Underpinning our analysis is the observation that for a significant class of MPI programs, the communication pattern is broadly input independent and therefore amenable to static analysis [5,6,11,22].…”

Discretionary Information Flow Control for Interaction-Oriented Specifications

“…There is indeed an urgent demand of formal guarantees for systems developed in such frameworks [18] and some flow analysis works already exist (e.g. [5,1]) also based on behavioural types (e.g. [17]).…”