Exploitation of Dynamic Communication Patterns through Static Analysis

Abstract

Abstract-Collective operations can have a large impact on the performance of parallel applications. However, the ideal implementation of a particular collective communication often depends on both the application and the targeted machine structure. Our approach combines dynamic and static analysis techniques to identify common collective communication patterns expressed as point-to-point calls and transforms them into equivalent MPI collectives. We first detect potential collective communication patterns in runtime traces and associate them with the corresponding source code regions. If our static analysis verifies that the introduction of collectives is safe for any program flow, we then replace the original communication primitives with their collective counterpart. In this paper we introduce the necessary algorithms to determine the safety of these transformations and we demonstrate several use cases, including automatic use of new extensions to the MPI standard such as nonblocking collective operations. The use of dynamic analysis significantly reduces compile times, resulting in a speed-up of about 50 for source transformations of HPL due to more directed analysis capabilities and also dramatically decreases complexity of the underlying static analysis.

Modernizing parallel code with pattern analysis

“…The technique has also been applied to synthetic benchmark generation, following the statistical profile of predefined pattern representatives [14]. Patterns can also be detected in legacy message-passing code [23,24,38]. A typical application is to find collective operations such as broadcasts, which the programmer has handcoded with point-to-point operations.…”

“…All of these projects use dynamic tracing to augment static analysis, and are consequently, like our work, programmer-advisory in nature. The work on collective operation finding in MPI programs discussed above [23,24,38] dynamically traces point-to-point message passing calls.…”

Towards Adapting Parallel Programs to Different Platforms: Identifying Interaction Patterns

“…Xu et al [6] identify the communication pattern and use this knowledge to perform event trace-compression with the purpose of obtaining application performance skeletons. Preissl et al [7], [8] present a method based on the ROSE framework [11], which was extended in [9] with the purpose of optimizing collective operations through static and dynamic analysis of applications. Although the above methods have proven efficient in identifying communication topologies, they all differ two-fold from the work presented here: first, our approach relies on identifying the communication patterns without prior execution of the code, and second, our method is network-centric, meaning that the knowledge we obtain on the communication topology is intended for a better mapping of processes to processing elements of the physical topology.…”