The use of mixture density networks in the emulation of complex epidemiological individual-based models

Abstract

Complex, highly-computational, individual-based models are abundant in epidemiology. For epidemics such as macro-parasitic diseases, detailed modelling of human behaviour and pathogen life-cycle are required in order to produce accurate results. This can often lead to models that are computationally-expensive to analyse and perform model fitting, and often require many simulation runs in order to build up sufficient statistics. Emulation can provide a more computationally-efficient output of the individual-based model, by approximating it using a statistical model. Previous work has used Gaussian processes (GPs) in order to achieve this, but these can not deal with multi-modal, heavy-tailed, or discrete distributions. Here, we introduce the concept of a mixture density network (MDN) in its application in the emulation of epidemiological models. MDNs incorporate both a mixture model and a neural network to provide a flexible tool for emulating a variety of models and outputs. We develop an MDN emulation methodology and demonstrate its use on a number of simple models incorporating both normal, gamma and beta distribution outputs. We then explore its use on the stochastic SIR model to predict the final size distribution and infection dynamics. MDNs have the potential to faithfully reproduce multiple outputs of an individual-based model and allow for rapid analysis from a range of users. As such, an open-access library of the method has been released alongside this manuscript. Author summaryInfectious disease modellers have a growing need to expose their models to a variety of stakeholders in interactive, engaging ways that allow them to explore different scenarios. This approach can come with a considerable computational cost that motivates providing a simpler representation of the complex model. We propose the use of mixture density networks as a solution to this problem. MDNs are highly flexible, deep neural networkbased models that can emulate a variety of data, including counts and over-dispersion. PLOS COMPUTATIONAL BIOLOGYPLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006869 March 16, 2020 1 / 16 OPEN ACCESS Citation: Davis CN, Hollingsworth TD, Caudron Q, Irvine MA (2020) The use of mixture density networks in the emulation of complex epidemiological individual-based models. PLoS Comput Biol 16(3): e1006869. https://doi.org/ 10.We explore their use firstly through emulating a negative binomial distribution, which arises in many places in ecology and parasite epidemiology. Then, we explore the approach using a stochastic SIR model. We also provide an accompanying Python library with code for all examples given in the manuscript. We believe that the use of emulation will provide a method to package an infectious disease model such that it can be disseminated to the widest audience possible. This is a PLOS Computational Biology Methods paper. PLOS COMPUTATIONAL BIOLOGYMixture-density network emulation of complex epidemiological individual-based models PLOS Computational Biology | https://doi...

Approximating solutions of the Chemical Master equation using neural networks

“…Our work differs from related approaches ( Gupta et al., 2021 ; Bortolussi and Palmieri, 2018 ; Repin and Petrov, 2021 ; Davis et al., 2020 ; Cairoli et al., 2021 ) in several regards. Unlike Bortolussi and Palmieri (2018) and Repin and Petrov (2021) or Cairoli et al.…”

“…Moreover, a number of recent studies have investigated the use of neural networks to learn various properties of stochastic biochemical reaction networks modeled using the CME ( Gupta et al., 2021 ; Bortolussi and Palmieri, 2018 ; Repin and Petrov, 2021 ; Davis et al., 2020 ; Cairoli et al., 2021 ). Schnoerr et al.…”

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

“…Similar to Lamperti's idea, Vahdati et al [156] built an ML surrogate model to map the relationship between the large parameter space of the microagent and simulation outcomes at the macrolevel. Davis et al [157] also used the ML technique to build the epidemic model with different parameters with the introduction of the concept of a mixture density network that incorporates a mixture model and an NN for emulating the statistical distribution of epidemiological models at the macrolevel, which can speed up the agent-based simulation for decision-making.…”

Optimizing COVID-19 vaccine distribution across the United States using deterministic and stochastic recurrent neural networks

“…MDNs are a combination of a neural network and a mixture of distributions, as represented in Fig 2 . In MDNs, neural networks are used to model a mixture of components [37]. The main aspects of MDNs include the type of neural network, the number and size of the hidden layers, the dimension of the output, the number of input parameters, the type of distribution, and the number of distributions [37]. Unlike the LSTM deterministic model with fully determined outputs, MDNs estimate probability distributions of potential outcomes.…”