MPI Pre-Processor: Generating MPI Derived Datatypes from C Datatypes Automatically

Abstract

The grid usage is facing the problem that consists in running existing sequential code for parallel execution transparently (ie. using source code without modification). AIPE is a middleware that deals with this problem. MPIPP is the software component we have developed to allow the generation of MPI derived datatypes from C datatype definitions automatically. The goal of this new tool is to make the building of complex messages easier for the end-user. Moreover, this paper shows that MPIPP goes farther in the complexity level of C datatypes that can be taken into account than any other similar tools have ever gone to.

Efficient automated marshaling of C++ data structures for MPI applications

“…However, a comparison of our work with other directly related approaches, such as MPI Pre-Processor [28], AutoMap/AutoLink [11], or C++2MPI [14], is impossible due to their lack of support for C++ language features or even C++ itself.…”

“…These approaches include both automatic tools and special purpose libraries. The automatic tools that we consider closely related work include AutoMap/AutoLink [11], C++2MPI [14], and MPI Pre-Processor [28]. In addition, Boost.MPI and Boost.Serialization [17] provide library support for seamless marshaling and unmarshaling.…”

“…C++2MPI [14] and MPI Pre-Processor [28] provide automatic creation of MPI Datatype's for C/C++ data structures. An MPI Datatype is a list of memory offsets describing the data to be marshaled given the base offset of a structure.…”

Exploiting points-to maps for de-/serialization code generation

“…However, implementation of de-/serialization methods of user defined classes is still realized manually with these frameworks. Second, there are tools for generating de-/serialization code, which parse the class structure and generate the functions accordingly [1,4,6,11,15,17,18]. However, these generators have restrictions on how arrays are implemented and do not support serialization of void/function pointers.…”

“…For each dynamic array attribute, some approaches [18,11,15,17,6] assume that there is a matching attribute storing its size. To be able to use these approaches for generating the de-/serialization methods for the computer architecture simulator, we need to make each element of array heapSize an attribute as shown in Figure 8 -(b).…”