The design of MPI based distributed shared memory systems to support OpenMP on clusters

Abstract

OpenMP can be supported in cluster environments by using distributed shared memory (DSM) systems. A portable approach for building DSM systems is to layer it on MPI. With these goals in mind, this paper makes two contributions. The first is a discussion about two software DSM systems that we have implemented using MPI. One uses background polling threads while the other uses processes that are driven only by incoming MPI messages. Comparisons of the two approaches show the latter to be a more scalable architecture that is better suited for the multi-core processors that are becoming commonplace. The second contribution recognizes that a common workaround for sub-team synchronizations in OpenMP is to use the flush directive on shared variables within busy-wait loops. In such a situation, only the flush in the last iteration of the busy-wait loop will result in the conditions necessary for exiting the loop. Thus transfer of the shared value need only be done if there were changes. We implement in our DSM a flush mechanism that eliminates the unnecessary data transfers entirely without any additional support or hints from the programmer.

An integrated, programming model-driven framework for NoC–QoS support in cluster-based embedded many-cores

“…The reason for these two programming paradigms lies in the nature of the memory subsystem of the target MPSoC, which is, in fact, a DSM platform. Hybrid OpenMP-MPI programming has proven beneficial for DSM systems in the HPC domain [17], and more in general to implement nested (multi-level) parallelism on NUMA systems [18]. Here, coarse-grained (task-level) parallelism can be mapped over different clusters communicating through MPI primitives, whereas fine-grained (loop-level) parallelism can be easily distributed within a cluster with OpenMP.…”

“…Overall, the platform consists of several clusters, interconnected with a top-level NoC. We assume a Distributed Shared Memory model (DSM) [17], where all processors in the system can directly access all the shared memory modules. However, due to the presence of the hierarchical interconnection system, the access latencies are subject to Non-Uniform Memory Access (NUMA) effects, depending on the physical distance of the end points.…”

Region-Based Prefetch Techniques for Software Distributed Shared Memory Systems

“…The total number of page faults (N f ) and the number of prefetches issued (N p ) are presented in thousands, and the number of useful prefetched pages is presented as a ratio to N f . The efficiency of each prefetch technique is calculated based on Equation (2).…”

“…The most widely used such paradigm, the shared memory programming model, can be facilitated on clusters via software Distributed Shared Memory (sDSM) systems, which have the advantage of both the programmability of shared memory programming models and the low cost of distributed memory architectures. sDSM systems designed with OpenMP compiler support are also known as cluster-enabled OpenMP systems [1], [2], [3], [4], [5].…”