DAPT: A Package Enabling Distributed Automated Parameter Testing

Abstract

Modern agent-based models (ABM) and other simulation models require evaluation and testing of many different parameters. Managing that testing for large scale parameter sweeps (grid searches) as well as storing simulation data requires multiple, potentially customizable steps that may vary across simulations. Furthermore, parameter testing, processing, and analysis are slowed if simulation and processing jobs cannot be shared across teammates or computational resources. While high-performance computing (HPC) has become increasingly available, models can often be tested faster through the use of multiple computers and HPC resources. To address these issues, we created the Distributed Automated Parameter Testing (DAPT) Python package. By hosting parameters in an online (and often free) "database", multiple individuals can run tests simultaneously in a distributed fashion, enabling ad hoc crowdsourcing of computational power. Combining this with a flexible, scriptable tool set, teams can evaluate models and assess their underlying hypotheses quickly. Here we describe DAPT and provide an example demonstrating its use.

DAPT: A package enabling distributed automated parameter testing

“…There are many basic examples provided with DAPT that demonstrate how each module functions. They can be found in the examples folder of the GitHub repository [8]. To provide a real-world example of how DAPT can be used, we will use PhysiCell [9] version 1.7.0, an open-source, agent-based multicellular simulation framework.…”

“…More information about the model along with a GUI to explore the parameters is available through the PhysiCell biorobots simulation tool on nanoHUB [10]. The code for this example can be found in the paper_example.py file in the DAPT example for this paper [11].…”

“…DAPT (RRID:SCR_021032) is primarily hosted on GitHub [14]. It is licensed under the BSD 3-clause license.…”

“…Other attributes can be included in the table to add additional information. For example, the attributes is found in the documentation [7]. To test stochastic models that require multiple runs at the same location in parameter space, parameter sets can be re-run in as many new, unique jobs as needed.…”

“…Then multiple team members can execute the script locally, completing the set of jobs concurrently. DAPT will determine which parameters each person should run when the next_parameter() method is called.The sheets_example.py script[12] in the PhysiCell example repository demonstrates how Google Sheets can be used as the database. Google Sheets Example Section of the README[13] outlines required changes to paper_example.py to enable use of Sheets and how to setup Google Sheets.…”