Improving Clairvoyant: reduction algorithm resilient to imbalanced process arrival patterns

Abstract

The Clairvoyant algorithm proposed in “A novel MPI reduction algorithm resilient to imbalances in process arrival times” was analyzed, commented and improved. The comments concern handling certain edge cases in the original pseudocode and description, i.e., adding another state of a process, improved cache friendliness more precise complexity estimations and some other issues improving the robustness of the algorithm implementation. The proposed improvements include skipping of idle loop rounds, simplifying generation of the ready set and management of the state array and an about 90-fold reduction in memory usage. Finally an extension enabling process arrival times (PATs) prediction was added: an additional background thread used to exchange the data with the PAT estimations. The performed tests, with a dedicated mini-benchmark executed in an HPC environment, showed correctness and improved performance of the solution, with comparison to the original or other state-of-the-art algorithms.

All-gather Algorithms Resilient to Imbalanced Process Arrival Patterns

“…Finally, we can observe that the algorithms are well scalable with the increasing numbers of the compute nodes. Similarly to our previous works [20,21,23,24], this approach can be directly used for HPC applications supported by iterative process model, such as various scientific simulations.…”

“…In both cases the theoretical analysis and experimental results showed the advantage of the newer approach. Recently, in Reference [23], we presented a new version of Clairvoyant algorithm, including a set of performance and usability improvements.…”

“…In our previous works [20,21,23,24], as well as in this article, we assume an iterative model of the processing, i.e., the parallel program executes consecutively iterations and each iteration consists of two phases: computations, where the processes perform calculations, and communication, with the data containing results of the computation phase being exchanged by the cooperating processes. The former causes heavy CPU load, in opposition to the latter when the network is intensively used.…”