ExaWorks: Workflows for Exascale

Al-Saadi, Aymen; Ahn, Dong H.; Babuji, Yadu; Chard, Kyle; Corbett, James C.; Hategan, Mihael; Herbein, Stephen; Jha, Shantenu; Laney, Dan; Merzky, André; Munson, Todd; Salim, Michael; Titov, Mikhail; Turilli, Matteo; Uram, Thomas D.; Wozniak, Justin M.

doi:10.1109/works54523.2021.00012

Cited by 15 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current era of big data and high-performance computing has emphasized the significant need for robust, flexible, and scalable workflow management solutions that can be used to efficiently orchestrate scientific calculations (Ben-Nun et al, 2020;Da Silva et al, 2023). To date, a wide variety of workflow systems have been developed, and it has become clear that there is no one-size-fits-all solution due to the diverse needs of the computational community (Al-Saadi et al, 2021;Existing Workflow Systems, 2023). While several popular software packages in this space have emerged over the last decade, many of them require the user to tailor their domain-specific code with the underlying workflow management framework closely in mind.…”

Section: Statement Of Needmentioning

confidence: 99%

Jobflow: Computational Workflows Made Simple

Rosen,

Gallant,

George

et al. 2024

JOSS

View full text Add to dashboard Cite

Section: Statement Of Needmentioning

confidence: 99%

Jobflow: Computational Workflows Made Simple

Rosen,

Gallant,

George

et al. 2024

JOSS

View full text Add to dashboard Cite

“…We also host an open monthly community meeting that is attended by representatives across DOE laboratories. A snapshot of progress and plans of ExaWorks was accepted for the SC21 workshop on workflows (WORKS21) [257].…”

Section: Recent Progressmentioning

confidence: 99%

ECP Software Technology Capability Assessment Report V3.0

Heroux

McInnes

et al. 2022

Self Cite

View full text Add to dashboard Cite

The Exascale Computing Project (ECP) Software Technology (ST) focus area is responsible for (1) developing critical software capabilities that will enable the successful execution of ECP applications and (2) providing key components of a productive and sustainable exascale computing ecosystem that will position the US Department of Energy (DOE) and the broader high-performance computing (HPC) community with a firm foundation for future extreme-scale computing capabilities.This ECP ST Capability Assessment Report (CAR) provides an overview and assessment of current ECP ST capabilities and activities, giving stakeholders and the broader HPC community information that can be used to assess ECP ST progress and plan their own efforts accordingly. ECP ST leaders commit to updating this document on regular basis (every 6-12 months). Highlights from this version of the report are presented here.This version of the CAR contains the following updates relative to the previous revision.• This report highlights the progress with the Extreme-scale Scientific Software Stack (E4S) efforts.In particular, this report discusses how E4S continues to gain traction as a first-class entity in the HPC ecosystem, enabling new conversations with users, facilities, vendors, other US agencies, and international partners.• The several-page summaries of each ECP Level 4 project were updated to reflect recent progress and next steps (Section 4). Of particular note are the experiences of our teams on early-access systems for Frontier.• The E4S is described further. E4S is now updated via quarterly releases. E4S is the primary integration and delivery vehicle for ECP ST capabilities (Section 2.1.1).• The ECP ST software development kit (SDK) effort further refined its groupings (Section 2.1.2).The ECP ST focus area represents the key bridge between exascale systems and the scientists developing applications that will run on those platforms. ECP ST efforts contribute to approximately 70 software products (Section 2.1.3) in six technical areas (Table 1). Since publishing the previous revision of the CAR, the team has continued to evolve the product dictionary of official product names, which enables more rigorous mapping of ECP ST deliverables to stakeholders (Section 2.1.4).Programming Models & Runtimes: In addition to developing key enhancements to MPI and OpenMP for scalable systems with accelerated node architectures, the team is working on performance portability layers (Kokkos and RAJA) and participating in OpenMP and OpenACC software design and development that will enable applications to write much of their source code without needing to provide vendor-specific implementations for each exascale system. One legacy of ECP ST efforts is anticipated to be a software stack that supports Intel and AMD accelerators in addition to NVIDIA's accelerators (Section 4.1).Development Tools: The team is enhancing existing widely used compilers (e.g., LLVM) and performance tools for next-generation platforms. Compilers are critical for heterogeneous archi...

show abstract