EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

Acer, Seher; Azad, Ariful; Boman, Erik G.; Buluç, Aydın; Devine, Karen Dragon; Ferdous, S. M.; Gawande, Nitin; Ghosh, Sayan; Halappanavar, Mahantesh; Kalyanaraman, Ananth; Khan, Arif; Minutoli, Marco; Pothen, Alex; Rajamanickam, Sivasankaran; Selvitopi, Oğuz; Tallent, Nathan R.; Tumeo, Antonino

doi:10.1177/10943420211029299

“…As the HPC and cloud computing communities increasingly rely on hardware specialization to improve performance, codesign approaches will support the development of accelerators (Lie 2021;Reuther et al 2021;Cortés et al 2021) for frequently used kernels in scientific modeling and AI/ML methods. New specialized accelerators may arise to support additional data science capabilities such as uncertainty quantification, streaming analytics, or graph analysis (Halappanavar et al 2021;Acer et al 2021). These future large-scale computing systems with extreme heterogeneity must be codesigned to support the increased computational and dataset sizes associated with Earth science predictability and scientific machine reasoning (Yang et al 2016;Zhang et al 2020;Yu et al 2022).…”

Section: Future System Conceptsmentioning

confidence: 99%

Perspectives on AI Architectures and Codesign for Earth System Predictability

Mudunuru,

Ang,

Halappanavar

et al. 2024

Artificial Intelligence for the Earth Systems

0

View full text Add to dashboard Cite

Recently, the U.S. Department of Energy (DOE), Office of Science, Biological and Environmental Research (BER), and Advanced Scientific Computing Research (ASCR) programs organized and held the Artificial Intelligence for Earth System Predictability (AI4ESP) workshop series. From this workshop, a critical conclusion that the DOE BER and ASCR community came to is the requirement to develop a new paradigm for Earth system predictability focused on enabling artificial intelligence (AI) across the field, lab, modeling, and analysis activities, called ModEx. The BER’s ‘Model-Experimentation’, ModEx, is an iterative approach that enables process models to generate hypotheses. The developed hypotheses inform field and laboratory efforts to collect measurement and observation data, which are subsequently used to parameterize, drive, and test model (e.g., process-based) predictions. A total of 17 technical sessions were held in this AI4ESP workshop series. This paper discusses the topic of the ‘AI Architectures and Co-design’ session and associated outcomes. The AI Architectures and Co-design session included two invited talks, two plenary discussion panels, and three breakout rooms that covered specific topics, including: (1) DOE high-performance computing (HPC) Systems, (2) Cloud HPC Systems, and (3) Edge computing and Internet of Things (IoT). We also provide forward-looking ideas and perspectives on potential research in this co-design area that can be achieved by synergies with the other 16 session topics. These ideas include topics such as: (1) reimagining co-design, (2) data acquisition to distribution, (3) heterogeneous HPC solutions for integration of AI/ML and other data analytics like uncertainty quantification with earth system modeling and simulation, and (4) AI-enabled sensor integration into earth system measurements and observations. Such perspectives are a distinguishing aspect of this paper.

show abstract

“…The authors have shown a 20Â speedup for the GraphSage benchmark (Hamilton et al, 2017) and a 36Â speedup on GAT benchmarks (Velickovic et al, 2018) In a more general setting, scaling of unstructured graph algorithms is an important problem. Within the Department of Energy, the ExaGraph Co-design Center is a component of the Exascale Computing Project, which aims to develop broadly applicable graph kernels for exascale computation (1 exaflop ¼ 10 18 flops) in applications spanning power grid, biology, chemistry, wind energy, and national security (Acer et al, 2021). The project aims to codesign algorithms, computing architecture, and hardware to build combinatorial kernels impacting general graph computation in addition to GNNs.…”

Section: Scalable Gnnsmentioning

confidence: 99%

Scalable algorithms for physics-informed neural and graph networks

Shukla

¹

,

Xu

²

,

Trask

³

et al. 2022

DCE

View full text Add to dashboard Cite

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.

show abstract

“…In a more general setting, scaling of unstructured graph algorithms is an important problem. Within the Department of Energy, the ExaGraph Co-design Center is a component of the Exascale Computing Project, which aims to develop broadly applicable graph kernels for exascale computation (1 exaflop = 10 18 flops) in applications spanning power grid, biology, chemistry, wind energy, and national security (Acer et al, 2021). The project aims to co-design algorithms, computing architecture, and hardware to build combinatorial kernels impacting general graph computation in addition to graph neural networks.…”

Section: Scalable Gnnsmentioning

confidence: 99%

Scalable algorithms for physics-informed neural and graph networks

Shukla¹,

Xu²,

Trask³

et al. 2022

Preprint

View full text Add to dashboard Cite

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space-time domain. Such physics-informed machine learning integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems. Impact StatementMany complex problems in computational engineering can be described by parametrized differential equations while boundary conditions or material properties may not be fully known, e.g., in thermal-fluid or solid mechanics systems. These ill-defined problems cannot be solved with standard numerical methods but if some sparse data are available, progress can be made by recasting them as large-scale minimization problems. Here, we review physics-informed neural networks (PINNs) and physics-informed graph networks (PIGNs) that integrate seamlessly data and mathematical physics models, even in partially understood or uncertain contexts. Using automatic differentiation in PINNs and external graph calculus in PIGNs, the physical laws are enforced by penalizing the residuals on random points in the space-time domain for PINNs and on the nodes of a graph in PIGNs. New multi-GPU algorithms are needed to scale up PINNs and PIGNs to realistic applications.

show abstract

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

Cited by 12 publications

References 100 publications

Perspectives on AI Architectures and Codesign for Earth System Predictability

Perspectives on AI Architectures and Codesign for Earth System Predictability

Scalable algorithms for physics-informed neural and graph networks

Scalable algorithms for physics-informed neural and graph networks

Contact Info

Product

Resources

About