Nonconforming Schwarz-spectral element methods for incompressible flow

Abstract

We present scalable implementations of spectral-element-based Schwarz overlapping (overset) methods for the incompressible Navier-Stokes (NS) equations. Our SEM-based overset grid method is implemented at the level of the NS equations, which are advanced independently within separate subdomains using interdomain velocity and pressure boundarydata exchanges at each timestep or sub-timestep. Central to this implementation is a general, robust, and scalable interpolation routine, gslib-findpts, that rapidly determines the computational coordinates (processor p, element number e, and local coordinates (r, s, t) ∈Ω := [−1, 1] 3 ) for any arbitrary point x * = (x * , y * , z * ) ∈ Ω ⊂ lR 3 . The communication kernels in gslib execute with at most log P complexity for P MPI ranks, have scaled to P > 10 6 , and obviate the need for development of any additional MPI-based code for the Schwarz implementation. The original interpolation routine has been extended to account for multiple overlapping domains. The new implementation discriminates the possessing subdomain by distance to the domain boundary, such that the interface boundary data is taken from the inner-most interior points. We present application of this approach to several heat transfer and fluid dynamic problems, discuss the computation/communication complexity and accuracy of the approach, and present performance measurements for P > 12, 000.

Efficient exascale discretizations: High-order finite element methods

“…We developed a characteristic-based spectral element method for moving-domain problems (Patel et al 2019), and demonstrated it for the TCC III engine model illustrated in Figure 18(b). We also added a significantly enhanced capability for handling complex moving geometries by adding scalable support for overset grids, referred to as NekNek, based on generalized Schwarz overlapping methods (Mittal et al, 2019). The NekNek multimesh coupling is based entirely on the kernels in Nek’s gslib communication library, which has scaled to millions of MPI ranks.…”

Scalable Interpolation on GPUs for Thermal Fluids Applications

“…Automatic generation of meshes with hex-elements can be difficult for certain complex geometries. One approach to simplifying the meshing process is to use a Schwarz overlapping method to divide the domain into multiple, easier to mesh regions [3]. Furthermore, dividing the domain into multiple meshes can simplify simulations that have moving components by meshing individual parts that only move relative to the other subdomains.…”

“…The overset grid support requires resolution of several other technical issues relating to mass conservation, timestepping, and parallelism which have been addressed for the SEM in [15,16] and in earlier efforts by [2,17]. For a GPU-based NSE solver, however, it is also critical to have fast interpolation on the GPU, such as provided by the routines described in the preceeding section.…”

“…There are many instances, however, where there is a distinct need to evaluate off grid quantities. Applications include overset grids [1,2,3], particle tracking (Sec. 5), and numerous solution interrogation techniques such as profile plotting and 2D slices through the domain for visualization or spectral analysis of turbulent flows.…”

Adaptive Surface Fitting and Tangential Relaxation for High-Order Mesh Optimization

“…These two key functionalities allow the transfer of a high-order FE function from one mesh onto another. The reader is referred to Section 2.3 of [27] for further details.…”