Towards Data-Flow Parallelization for Adaptive Mesh Refinement Applications

Abstract

Adaptive Mesh Refinement (AMR) is a prevalent method used by distributed-memory simulation applications to adapt the accuracy of their solutions depending on the turbulent conditions in each of their domain regions. These applications are usually dynamic since their domain areas are refined or coarsened in various refinement stages during their execution. Thus, they periodically redistribute their workloads among processes to avoid load imbalance. Although the defacto standard for scientific computing in distributed environments is MPI, in recent years, pure MPI applications are being ported to hybrid ones, attempting to cope with modern multi-core systems. Recently, the Task-Aware MPI library was proposed to efficiently integrate MPI communications and tasking models, providing also the transparent management of communications issued by tasks. In this paper, we demonstrate the benefits of porting AMR applications to data-flow programming models leveraging that novel hybrid approach. We exploit most of the application parallelism by taskifying all stages, allowing their natural overlap. We employ these techniques on the miniAMR proxy application, which mimics the refinement, load balancing, communication, and computation patterns of general AMR applications. We evaluate how this approach reduces the time in its computation and communication phases while achieving better programmability than other conventional hybrid techniques.

Combining One-Sided Communications with Task-Based Programming Models

“…The hybrid variants send/receive/write each boundary block face from a different task (separate messages). That is not the optimal configuration (the optimal is around eight faces per message) but provides very reasonable performance [19] and puts more pressure on the communication phases. We show the throughput speedup of the strong scaling on the upper part of Figure 11 and the parallel efficiency on the lower part.…”

“…MiniAMR features multiple phases of computation and communication interleaved, and then a refinement and load-balancing phase periodically. Previous works [24] [19] fully taskified its computation and communication phases and some parts of the refinement and load-balancing [19], using OmpSs-2 and TAMPI. We take that taskification as the base and port the communication phases to TAGASPI.…”

“…We run the following experiments in Marenostrum4; the MPI-only uses 48 ranks/node, and hybrids use 4 ranks/node and 12 cores/rank. That is the optimal configuration for hybrid approaches in miniAMR, given that the refinement phase is not fully taskified [19]. In the TAGASPI variant, we also use TAMPI during the load-balancing stage to demonstrate that both libraries can work together.…”

“…Firstly, we perform a strong scaling experiment with the same input used in the previous study of miniAMR [19], but we halve the number of computed variables (to 20 variables) to reduce the computational weight. The hybrid variants send/receive/write each boundary block face from a different task (separate messages).…”

Combining Dynamic Concurrency Throttling with Voltage and Frequency Scaling on Task-based Programming Models

“…The set of experiments we used includes well-known benchmarks such as Cholesky, Dotproduct, MultiSAXPY, STREAM, NBody, NQueens, and the Gauss-Seidel solver for the Heat The benchmarks of this list can be categorized as (i) purely memory-bounded, such as Heat, Dotproduct, MultiSAXPY, and STREAM, (ii) purely compute-bounded, such as NBody and NQueens, and (iii) balanced, such as Cholesky, miniAMR, HPCCG, and LULESH. Furthermore, all of these benchmarks have been parallelized using tasks, as their task-based versions [15], [16], [17] offer competitive or better performance than the fork-join OpenMP counterpart. Finally, the evaluation is partitioned into two phases.…”