Extreme-Scale Dynamic Exploration of a Distributed Agent-Based Model With the EMEWS Framework

Abstract

Agent-based models (ABMs) integrate multiple scales of behavior and data to produce higher-order dynamic phenomena and are increasingly used in the study of important social complex systems in biomedicine, socio-economics and ecology/resource management. However, the development, validation and use of ABMs is hampered by the need to execute very large numbers of simulations in order to identify their behavioral properties, a challenge accentuated by the computational cost of running realistic, large-scale, potentially distributed ABM simulations. In this paper we describe the Extreme-scale Model Exploration with Swift (EMEWS) framework, which is capable of efficiently composing and executing large ensembles of simulations and other “black box” scientific applications while integrating model exploration (ME) algorithms developed with the use of widely available 3rd-party libraries written in popular languages such as R and Python. EMEWS combines novel stateful tasks with traditional run-to-completion many task computing (MTC) and solves many problems relevant to high-performance workflows, including scaling to very large numbers (millions) of tasks, maintaining state and locality information, and enabling effective multiple-language problem solving. We present the high-level programming model of the EMEWS framework and demonstrate how it is used to integrate an active learning ME algorithm to dynamically and efficiently characterize the parameter space of a large and complex, distributed Message Passing Interface (MPI) agent-based infectious disease model.

Learning-accelerated Discovery of Immune-Tumour Interactions

“…This process is continued until a convergence or, in the present case, a maximum number of iterations is achieved. Details of the AL algorithm are further described in 36 and the R code can be found in GitHub; see the link in the Electronic Supplementary Material.…”

“…As previously noted, the parameter spaces of complex ABMs, coupled with the highly non-linear relationship between ABM input parameters and model outputs, require heuristic ME approaches that adaptively evaluate large numbers of simulations. Here we give a brief overview of how the Extremescale Model Exploration with Swift (EMEWS) framework 30 enables the creation of HPC workflows for implementing largescale ME studies (see 36 for further details).…”

Learning-accelerated discovery of immune-tumour interactions

“…As previously noted, the parameter spaces of complex ABMs, coupled with the highly non-linear relationship between ABM input parameters and model outputs, require heuristic ME approaches that adaptively evaluate large numbers of simulations. Here we give a brief overview of how the Extreme-scale Model Exploration with Swift (EMEWS) framework31 enables the creation of HPC workflows for implementing large-scale ME studies (see ref. 37 for further details).…”

Nested Active Learning for Efficient Model Contextualization and Parameterization: Pathway to generating simulated populations using multi-scale computational models

“…Our ML models are trained and integrated using the Extreme-scale Model Exploration With Swift (EMEWS) framework (21)(22)(23). EMEWS enables the creation of high-performance computing (HPC) workflows for implementing large-scale model exploration studies.…”