Heterogeneous parallel algorithm design and performance optimization for WENO on the Sunway Taihulight supercomputer

Abstract

A Weighted Essentially Non-Oscillatory scheme (WENO) is a solution to hyperbolic conservation laws, suitable for solving high-density fluid interface instability with strong intermittency. These problems have a large and complex flow structure. To fully utilize the computing power of High Performance Computing (HPC) systems, it is necessary to develop specific methodologies to optimize the performance of applications based on the particular system's architecture. The Sunway TaihuLight supercomputer is currently ranked as the fastest supercomputer in the world. This article presents a heterogeneous parallel algorithm design and performance optimization of a high-order WENO on Sunway TaihuLight. We analyzed characteristics of kernel functions, and proposed an appropriate heterogeneous parallel model. We also figured out the best division strategy for computing tasks, and implemented the parallel algorithm on Sunway TaihuLight. By using access optimization, data dependency elimination, and vectorization optimization, our parallel algorithm can achieve up to 172× speedup on one single node, and additional 58× speedup on 64 nodes, with nearly linear scalability.

An MPI+OpenACC-based PRM scalar advection scheme in the GRAPES model over a cluster with multiple CPUs and GPUs

“…Further, the code needs fewer changes because of the high-level programming model of OpenACC and is kept understandable and more uncomplicated by the programmer; moreover, single source code can be used to target both CPU only and hybrid CPU-GPU architectures, as the directives can be ignored when no GPU is available in the system. Several investigations have shown that the performance loss with OpenACC is due to the lack of support of sharing memory usage and banking conflicts, which is often not substantial, particularly on the most recent GPUs [11,15] . We employed the hybrid MPI and OpenACC programming method.…”

“…Several examples that partially adapted GPUs in weather and climate prediction codes showed performance gains [7][8][9][10][11][12][13][14][15][16][17] . Especially, GPU acceleration of scalar or tracer advection modules using Compute Unified Device Architecture (CUDA) C/Fortran achieves an approximately three-fold speedup [12,18,19] .…”

Multi-agent modeling and simulation in the AI age

“…Numerous approaches have emerged in various fields trying to solve this problem. Among them are two most effective and common solutions: One is to restructure the model at the software level by super individual or other methods; another is to speed up large-scale computing by distributed parallel computing [91,92] or using new computation tools, such as the Quantum tool [93] .…”

SignGD with error feedback meets lazily aggregated technique: Communication-efficient algorithms for distributed learning

“…The past few decades have witnessed an explosion of data in both the number of observations and parameters, resulting in significant interests in distributed algorithms for solving large-scale machine learning problems [1][2][3][4][5][6][7] . However, efficient implementations of the distributed optimization algorithms for machine learning applications are challenging.…”