Progress Thread Placement for Overlapping MPI Non-blocking Collectives Using Simultaneous Multi-threading

Abstract

Non-blocking collectives have been proposed so as to allow communications to be overlapped with computation in order to amortize the cost of MPI collective operations. To obtain a good overlap ratio, communications and computation have to run in parallel. To achieve this, different hardware and software techniques exists. Dedicated some cores to run progress threads is one of them. However, some CPUs provide Simultaneous Multi-Threading, which is the ability for a core to have multiple hardware threads running simultaneously, sharing the same arithmetic units. Our idea is to use them to run progress threads to avoid dedicated cores allocation. We have run benchmarks on Haswell processors, using its Hyper-Threading capability, and get good results for both performance and overlap only when inter-node communications are used by MPI processes. However, we also show that enabling Simultaneous Multi-Threading for intra-communications leads to bad performances due to cache effects.

Reactive Task Migration for Hybrid MPI+OpenMP Applications

“…We found that using a dedicated core is essential to guarantee sufficient progression of MPI messages and achieving our objective of fine-granular reactivity. Similar findings have been reported in [4,6]. In contrast to predictive load balancing, there is no mutual a-priori agreement on the task migration pattern.…”

Revisiting performance of BiCGStab methods for solving systems with multiple right-hand sides

“…This option is typically implemented with threads, which handle the status of the non-blocking operations and perform the corresponding progression. The drawback for this strategy is related with significant overhead, produced by the progression threads [26,27,28,29]. The manual progression is generally independent on the hardware and MPI library implementation, but needs some user efforts to add MPI Test or MPI Probe calls to progress the communications.…”