Solving global shallow water equations on heterogeneous supercomputers

Abstract

The scientific demand for more accurate modeling of the climate system calls for more computing power to support higher resolutions, inclusion of more component models, more complicated physics schemes, and larger ensembles. As the recent improvements in computing power mostly come from the increasing number of nodes in a system and the integration of heterogeneous accelerators, how to scale the computing problems onto more nodes and various kinds of accelerators has become a challenge for the model development. This paper describes our efforts on developing a highly scalable framework for performing global atmospheric modeling on heterogeneous supercomputers equipped with various accelerators, such as GPU (Graphic Processing Unit), MIC (Many Integrated Core), and FPGA (Field Programmable Gate Arrays) cards. We propose a generalized partition scheme of the problem domain, so as to keep a balanced utilization of both CPU resources and accelerator resources. With optimizations on both computing and memory access patterns, we manage to achieve around 8 to 20 times speedup when comparing one hybrid GPU or MIC node with one CPU node with 12 cores. Using a customized FPGA-based data-flow engines, we see the potential to gain another 5 to 8 times improvement on performance. On heterogeneous supercomputers, such as Tianhe-1A and Tianhe-2, our framework is capable of achieving ideally linear scaling efficiency, and sustained double-precision performances of 581 Tflops on Tianhe-1A (using 3750 nodes) and 3.74 Pflops on Tianhe-2 (using 8644 nodes). Our study also provides an evaluation on the programming paradigm of various accelerator architectures (GPU, MIC, FPGA) for performing global atmospheric simulation, to form a picture about both the potential performance benefits and the programming efforts involved.

Application of particle swarm optimization in optimal placement of tsunami sensors

“…They are often used to describe geophysical flows (such as rivers, lakes, coastal areas, etc. ), rainfall runoff, tsunami propagation, or even atmospheric flows, given the appropriate initial conditions and source terms (Backhaus, 1983;Cea & Bladé, 2015;Rahman & Mandokhail, 2018;Galewsky, Scott & Polvani, 2004;Fu et al, 2017;Liu et al, 2008Liu et al, , 1995. SWE are derived from the Navier-Stokes equations, with the main assumption that the horizontal length scale is much greater than the vertical length scale.…”

A Hardware/Software Codesign for the Chemical Reactivity of BRAMS

“…In Fu et al (2017), they implement Shallow Water Equations solver in three different architectures. One of them is CPU-FPGA, they had to decompose the solver in three FPGAs to fit the double precision version; this solution resulted in a low bandwidth among the FPGAs.…”

High-performance Shallow Water Model for Use on Massively Parallel and Heterogeneous Computing Systems

“…General-purpose computing on GPUs is becoming increasingly popular for climate modeling too. There are examples of successful porting of atmosphere and ocean models on GPUs, including shallow water models [12,18,25]. We present three hybrid parallel programming patterns for calculations on GPUs: hybrid MPI-CUDA, hybrid asynchronous MPI-OpenMP-CUDA, and multi GPUs per node MPI-OpenMP-CUDA calculation patterns for effective use on heterogeneous computing systems.…”