Evaluation of dataflow programming models for electronic structure theory

Abstract

Summary Dataflow programming models have been growing in popularity as a means to deliver a good balance between performance and portability in the post‐petascale era. In this paper, we evaluate different dataflow programming models for electronic structure methods and compare them in terms of programmability, resource utilization, and scalability. In particular, we evaluate two programming paradigms for expressing scientific applications in a dataflow form: (1) explicit dataflow, where the dataflow is specified explicitly by the developer, and (2) implicit dataflow, where a task scheduling runtime derives the dataflow using per‐task data‐access information embedded in a serial program. We discuss our findings and present a thorough experimental analysis using methods from the NWChem quantum chemistry application as our case study, and OpenMP, StarPU, and PaRSEC as the task‐based runtimes that enable the different forms of dataflow execution. Furthermore, we derive an abstract model to explore the limits of the different dataflow programming paradigms.

Parallel and distributed algorithms

“…Dataflow programming models are becoming popular as a means to deliver a good balance between performance and portability in the post‐petascale era. The second paper, Evaluation of dataflow programming models for electronic structure theory, evaluates two programming paradigms for expressing scientific applications in a dataflow form: “explicit dataflow” where the dataflow is specified explicitly by the developer and “implicit dataflow” where a task scheduling runtime derives the dataflow using per‐task data‐access information embedded in a serial program. The paper compares these two models in terms of programmability, resource utilization, and scalability.…”