High-order asynchrony-tolerant finite difference schemes for partial differential equations

Abstract

Synchronizations of processing elements (PEs) in massively parallel simulations, which arise due to communication or load imbalances between PEs, significantly affect the scalability of scientific applications. We have recently proposed a method based on finite-difference schemes to solve partial differential equations in an asynchronous fashion -synchronization between PEs is relaxed at a mathematical level. While standard schemes can maintain their stability in the presence of asynchrony, their accuracy is drastically affected. In this work, we present a general methodology to derive asynchrony-tolerant (AT) finite difference schemes of arbitrary order of accuracy, which can maintain their accuracy when synchronizations are relaxed. We show that there are several choices available in selecting a stencil to derive these schemes and discuss their effect on numerical and computational performance. We provide a simple classification of schemes based on the stencil and derive schemes that are representative of different classes. Their numerical error is rigorously analyzed within a statistical framework to obtain the overall accuracy of the solution. Results from numerical experiments are used to validate the performance of the schemes.

Direct Numerical Simulations of turbulent flows using high-order Asynchrony-Tolerant schemes: accuracy and performance

“…This prevents the use of the standard finite difference schemes asynchronously and necessitates the need for numerical methods that are resilient to asynchrony. Such family of schemes has been put forth in [24]. These so-called Asynchrony-Tolerant (AT) schemes preserve the order of accuracy, despite asynchrony and are described next.…”

“…Here the weights c l m 's are computed by solving a system of linear equations constructed by imposing order of accuracy constraints on the Taylor series expansion of u n−l i+m in space and time. The choice of stencil and the general methodology for the derivation of these AT schemes has been explained in detail in [24]. As an example, a second order AT scheme at the left boundary Eq.…”

“…For M = 1, corresponding to an AT scheme with leading truncation error term of order O(∆x a ) where a = 2 when ∆t ∼ ∆x 2 [24], and a domain decomposed into 2 PEs such that PE (1) holds gridpoints i ∈ [1, N/2] and PE (2) holds gridpoints i ∈ [N/2 + 1, N ] and satisfies periodic boundary conditions, we can write Eq. ( 13) as…”

“…Not only can AB be efficiently implemented with only six communications per PE per time-step, it only requires computation of f n every time-step since f n−m , m > 0 is used from previous steps. Furthermore, the computation of f n−m using AT schemes does not alter the order of accuracy of AB schemes [24]. Thus, here we use second-order AB schemes for the temporal evolution in both synchronous and asynchronous simulations.…”

“…Early work in the literature dealt with asynchronous simulations but severely limited to lower orders of accuracy and restricted to certain class of PDEs [19,20,21,22]. A new and more generalized approach, extensible to arbitrarily high orders of accuracy, has been recently developed [23,24] to derive the so-called Asynchrony-Tolerant (AT) finite-difference schemes.…”