Added Concurrency to Improve MPI Performance on Multicore

Abstract

Abstract-MPI implementations typically equate an MPI process with an OS-process, resulting in a coarse-grain programming model where MPI processes are bound to the physical cores. FineGrain (FG-MPI) extends the MPICH2 implementation of MPI and implements an integrated runtime system to allow multiple MPI processes to execute concurrently inside an OS-process.FG-MPI's integrated approach makes it possible to add more concurrency than available parallelism, while minimizing the overheads related to context switches, scheduling and synchronization. In this paper we evaluate the benefits of added concurrency for cache awareness and message size and show that performance gains are possible by using FG-MPI to adjust the grain-size of a program to better fit the cache and potential advantages in passing smaller versus larger messages.We evaluate the use of FG-MPI on the complete set of the NAS parallel benchmarks over large problem sizes, where we show significant performance improvement (20%-30%) for three of the eight benchmarks. We discuss the characteristics of the benchmarks with regards to trade-offs between the added costs and benefits.

Checkpoint/restart approaches for a thread-based MPI runtime

“…Another work describes how, even if possibly harmful inside one application, oversubscribing can be used to efficiently execute multiple applications sharing one node [37]. To circumvent this drawback when applying MPI oversubscribing in a unique application, some work focused on enabling multiple MPI process in one OS process [26], verifying the positive impact of such implementation.…”

A parallel processing model for big medical image data

“…Computer cluster can be considered as an implementation of parallel computing, which can be viewed as a single system in many respects, overcoming the limited performance of a single computer and improving performance. What is more, optimization of performance of cluster or parallel computing has never been stopped, such as using CUDA inter-process communication [1], adding concurrency [2] and using dynamic energy optimization method of network [3] to improve MPI performance on the cluster.…”