Parallel Memory-Efficient Adaptive Mesh Refinement on Structured Triangular Meshes with Billions of Grid Cells

Abstract

We present sam(oa) 2 , a software package for a dynamically adaptive, parallel solution of 2D partial differential equations on triangular grids created via newest vertex bisection. An element order imposed by the Sierpinski space-filling curve provides an algorithm for grid generation, refinement, and traversal that is inherently memory efficient. Based purely on stack and stream data structures, it completely avoids random memory access. Using an element-oriented data view suitable for local operators, concrete simulation scenarios are implemented based on control loops and event hooks, which hide the complexity of the underlying traversal scheme. Two case studies are presented: two-phase flow in heterogeneous porous media and tsunami wave propagation, demonstrated on the Tohoku tsunami 2011 in Japan. sam(oa) 2 features hybrid MPI+OpenMP parallelization based on the Sierpinski order induced on the elements. Sections defined by contiguous grid cells define atomic tasks for OpenMP work sharing and stealing, as well as for migration of grid cells between MPI processes. Using optimized communication and load balancing algorithms, sam(oa) 2 achieves 88% strong scaling efficiency from 16 to 512 cores and 92% efficiency in a weak scaling test on 8,192 cores with 10 billion elements—all tests including adaptive mesh refinement and load balancing in each time step.

Accelerating an Adaptive Mesh Refinement Code for Depth‐Averaged Flows Using GPUs

“…The last variant is a combination of the first two, often referred to as block-based AMR. Unlike the patch-based AMR, which stores the multiresolution grid hierarchy as overlapping and nested grid patches, this approach stores the grid hierarchy as nonoverlapping fixed-size grid patches, each of which is stored as a leaf in a forest of quadtrees or octrees (e.g., Burstedde et al, 2011Burstedde et al, , 2014Fryxell et al, 2000;Liang & Borthwick, 2009;Liang et al, 2004;MacNeice et al, 2000;Meister et al, 2017;Popinet, 2012). Other multiresolution approaches, such as adaptive wavelet methods, have also been applied to shallow water equations, (e.g., Aechtner et al, 2015).…”

Methods and Test Cases for Linking Physics-Based Earthquake and Tsunami Models

“…Adaptive mesh refinement can enable the efficient computation of large domains, while at the same time it allows for high local resolution and geometric accuracy. This numerical scheme has been recently integrated into the adaptive mesh refinement package sam(oa) 2 (Meister et al 2016) (https://gitlab.lrz.de/samoa/samoa). sam(oa) 2 features efficient adaptive mesh refinement for tree-structured triangular meshes and provides parallelization in shared (using OpenMP) and distributed (via MPI) memory.…”

“…sam(oa) 2 features efficient adaptive mesh refinement for tree-structured triangular meshes and provides parallelization in shared (using OpenMP) and distributed (via MPI) memory. It has been shown to scale up to thousands of compute cores, with problem sizes that exceed one billion grid cells with dynamic adaptive refinement and coarsening of cells (Meister et al 2016).…”

Unstructured mesh generation based on Parallel Virtual Machine in cyber-physical system

“…Hence, it is necessary to further improve the parallel grid generation technique [20][21][22] to solve flow problem with complex geometries [23][24][25]. It is well known that CFD is becoming increasingly sophisticated: grids define highly complex geometries and flows are simulated involving very different length and time scales [26,27]. The number of grid points, and thereby the number of degrees of freedom, is increasing as the memory of supercomputers is growing.…”