Parallel asynchronous algorithms: A survey

Spitéri, Pierre

doi:10.1016/j.advengsoft.2020.102896

Cited by 19 publications

(13 citation statements)

References 111 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mathematical models of asynchronous iterations were developed in order to study their convergence; see, e.g., [1,3,5,22,34,36]. Sometimes these models are referred to as mathematical descriptions of computational models of asynchronous iterations.…”

Section: General Asynchronous Iterationsmentioning

confidence: 99%

See 1 more Smart Citation

Asynchronous domain decomposition methods for nonlinear PDEs

Chaouqui¹,

Chow²,

Szyld³

2022

etna

View full text Add to dashboard Cite

One-and two-level parallel asynchronous methods for the numerical solution of nonlinear systems of equations, especially those arising from (nonlinear) partial differential equations, are studied. The proposed methods are based on domain decomposition techniques. Local convergence theorems are presented in several cases, with appropriate hypotheses. Computational results on a shared memory multiprocessor machine for various problems exhibiting nonlinearities are reported, illustrating the potential of these asynchronous methods, especially for heterogeneous clusters.

show abstract

Section: General Asynchronous Iterationsmentioning

confidence: 99%

“…The last one indicates that no processor stops being updated. Convergence results for (3.1) were discussed and analyzed in several papers; see, e.g., [1,22,34] and references therein.…”

Section: General Asynchronous Iterationsmentioning

confidence: 99%

Asynchronous domain decomposition methods for nonlinear PDEs

Chaouqui¹,

Chow²,

Szyld³

2022

etna

View full text Add to dashboard Cite

show abstract

“…second-order perturbation, there are up-to (2) = 1, 261 state-space equations; they become up-to (3) = 44, 136 for = 3 and up-to (4) = 1, 544, 761 for = 4. It is noteworthy here that: ) the actual number of 'dominant' random variables can be significantly less; ) parallelisation and domain decomposition methods [45,46] can be used to simultaneously solve multiple UP equations, which in turn can drastically reduce the computational time.…”

Section: Quantification Of Response Statisticsmentioning

confidence: 99%

A reduced modal subspace approach for damped stochastic dynamic systems

Kasinos

Palmeri

Lombardo

et al. 2021

Computers & Structures

View full text Add to dashboard Cite

“…A class of numerical algorithms that by design have properties attractive for resilience are asynchronous iterative methods (Baudet, 1978; Baz et al, 1996; Bertsekas, 1983; Bertsekas and Tsitsiklis, 1989; Frommer and Szyld, 2000; Spitéri, 1986; Spiteri, 2020; Szyld, 1998). In order to avoid misunderstandings, we point out that this class of methods is unrelated to the idea of asynchronous dynamic load balancing (Klawonn et al, 2020) as addressed in Section 2.1.…”

Section: Numerical Algorithms For Resiliencementioning

confidence: 99%

Resiliency in numerical algorithm design for extreme scale simulations

Agullo

Altenbernd

Anzt

et al. 2021

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

This work is based on the seminar titled ‘Resiliency in Numerical Algorithm Design for Extreme Scale Simulations’ held March 1–6, 2020, at Schloss Dagstuhl, that was attended by all the authors. Advanced supercomputing is characterized by very high computation speeds at the cost of involving an enormous amount of resources and costs. A typical large-scale computation running for 48 h on a system consuming 20 MW, as predicted for exascale systems, would consume a million kWh, corresponding to about 100k Euro in energy cost for executing 1023 floating-point operations. It is clearly unacceptable to lose the whole computation if any of the several million parallel processes fails during the execution. Moreover, if a single operation suffers from a bit-flip error, should the whole computation be declared invalid? What about the notion of reproducibility itself: should this core paradigm of science be revised and refined for results that are obtained by large-scale simulation? Naive versions of conventional resilience techniques will not scale to the exascale regime: with a main memory footprint of tens of Petabytes, synchronously writing checkpoint data all the way to background storage at frequent intervals will create intolerable overheads in runtime and energy consumption. Forecasts show that the mean time between failures could be lower than the time to recover from such a checkpoint, so that large calculations at scale might not make any progress if robust alternatives are not investigated. More advanced resilience techniques must be devised. The key may lie in exploiting both advanced system features as well as specific application knowledge. Research will face two essential questions: (1) what are the reliability requirements for a particular computation and (2) how do we best design the algorithms and software to meet these requirements? While the analysis of use cases can help understand the particular reliability requirements, the construction of remedies is currently wide open. One avenue would be to refine and improve on system- or application-level checkpointing and rollback strategies in the case an error is detected. Developers might use fault notification interfaces and flexible runtime systems to respond to node failures in an application-dependent fashion. Novel numerical algorithms or more stochastic computational approaches may be required to meet accuracy requirements in the face of undetectable soft errors. These ideas constituted an essential topic of the seminar. The goal of this Dagstuhl Seminar was to bring together a diverse group of scientists with expertise in exascale computing to discuss novel ways to make applications resilient against detected and undetected faults. In particular, participants explored the role that algorithms and applications play in the holistic approach needed to tackle this challenge. This article gathers a broad range of perspectives on the role of algorithms, applications and systems in achieving resilience for extreme scale simulations. The ultimate goal is to spark novel ideas and encourage the development of concrete solutions for achieving such resilience holistically.

show abstract

Parallel asynchronous algorithms: A survey

Cited by 19 publications

References 111 publications

Asynchronous domain decomposition methods for nonlinear PDEs

Asynchronous domain decomposition methods for nonlinear PDEs

A reduced modal subspace approach for damped stochastic dynamic systems

Resiliency in numerical algorithm design for extreme scale simulations

Contact Info

Product

Resources

About