Improving Scalability of Silent-Error Resilience for Message-Passing Solvers via Local Recovery and Asynchrony

Kolla, Hemanth; Mayo, Jackson; Teranishi, Keita; Armstrong, Robert C.

doi:10.1109/ftxs51974.2020.00006

Cited by 3 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AT schemes, which relax synchronization at a mathematical level, can be coupled with such data-centric programming models to create highly scalable PDE solvers. Asynchrony has also been utilized in [62] for scalable resilience to soft faults. However, in [62] all derivatives are still computed with the most updated data but the computations are re-arranged to ensure maximum overlap between communication and computation.…”

Section: Discussionmentioning

confidence: 99%

“…Asynchrony has also been utilized in [62] for scalable resilience to soft faults. However, in [62] all derivatives are still computed with the most updated data but the computations are re-arranged to ensure maximum overlap between communication and computation. The asynchronization approach utilized in [62] coupled with mathematical level asynchrony with AT schemes can be effective in pushing the scaling wall.…”

Section: Discussionmentioning

confidence: 99%

“…However, in [62] all derivatives are still computed with the most updated data but the computations are re-arranged to ensure maximum overlap between communication and computation. The asynchronization approach utilized in [62] coupled with mathematical level asynchrony with AT schemes can be effective in pushing the scaling wall. Furthermore, since AT schemes do not require updated data, these schemes can also be used to recover from node failures without halting the simulation altogether.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of finite difference based asynchronous partial differential equations solver for reacting flows

Кумари

Cleary

Donzis

et al. 2023

Journal of Computational Physics

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of finite difference based asynchronous partial differential equations solver for reacting flows

Кумари

Cleary

Donzis

et al. 2023

Journal of Computational Physics

View full text Add to dashboard Cite

Task-Level Checkpointing and Localized Recovery to Tolerate Permanent Node Failures for Nested Fork–Join Programs in Clusters

Reitz,

Fohry

2024

SN COMPUT. SCI.

View full text Add to dashboard Cite

Exascale supercomputers consist of millions of processing units, and this number is still growing. Therefore, hardware failures, such as permanent node failures, become increasingly frequent. They can be tolerated with system-level Checkpoint/Restart, which saves the whole application state transparently and, if needed, restarts the application from the saved state; or with application-level checkpointing, which saves only relevant data via explicit calls in the program. The former approach requires no additional programming expense, whereas the latter is more efficient and allows to continue program execution after failures on the intact resources (localized shrinking recovery). An increasingly popular programming paradigm is asynchronous many-task (AMT) programming. Here, programmers identify parallel tasks, and a runtime system assigns the tasks to worker threads. Since tasks have clearly defined interfaces, the runtime system can automatically extract and save their interface data. This approach, called task-level checkpointing (TC), combines the respective strengths of system-level and application-level checkpointing. AMTs come in many variants, and so far, TC has only been applied to a few, rather simple variants. This paper considers TC for a different AMT variant: nested fork–join (NFJ) programs that run on clusters of multicore nodes under work stealing. We present the first TC scheme for this setting. It performs a localized shrinking recovery and can handle multiple node failures. In experiments with four benchmarks, we observed execution time overheads of around 44 % at 1536 workers, and negligible recovery costs. Additionally, we developed and experimentally validated a prediction model for the running times of the scheme.

show abstract

Asynchrony and Failure Masking via Pseudo-Local Process Recovery in MPI Applications

Whitlock,

Kolla,

Bouteiller

et al. 2024

2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

View full text Add to dashboard Cite

Improving Scalability of Silent-Error Resilience for Message-Passing Solvers via Local Recovery and Asynchrony

Cited by 3 publications

References 19 publications

Evaluation of finite difference based asynchronous partial differential equations solver for reacting flows

Evaluation of finite difference based asynchronous partial differential equations solver for reacting flows

Task-Level Checkpointing and Localized Recovery to Tolerate Permanent Node Failures for Nested Fork–Join Programs in Clusters

Asynchrony and Failure Masking via Pseudo-Local Process Recovery in MPI Applications

Contact Info

Product

Resources

About