Assessing parameter identifiability in compartmental dynamic models using a computational approach: application to infectious disease transmission models

Abstract

BackgroundMathematical modeling is now frequently used in outbreak investigations to understand underlying mechanisms of infectious disease dynamics, assess patterns in epidemiological data, and forecast the trajectory of epidemics. However, the successful application of mathematical models to guide public health interventions lies in the ability to reliably estimate model parameters and their corresponding uncertainty. Here, we present and illustrate a simple computational method for assessing parameter identifiability in compartmental epidemic models.MethodsWe describe a parametric bootstrap approach to generate simulated data from dynamical systems to quantify parameter uncertainty and identifiability. We calculate confidence intervals and mean squared error of estimated parameter distributions to assess parameter identifiability. To demonstrate this approach, we begin with a low-complexity SEIR model and work through examples of increasingly more complex compartmental models that correspond with applications to pandemic influenza, Ebola, and Zika.ResultsOverall, parameter identifiability issues are more likely to arise with more complex models (based on number of equations/states and parameters). As the number of parameters being jointly estimated increases, the uncertainty surrounding estimated parameters tends to increase, on average, as well. We found that, in most cases, R0 is often robust to parameter identifiability issues affecting individual parameters in the model. Despite large confidence intervals and higher mean squared error of other individual model parameters, R0 can still be estimated with precision and accuracy.ConclusionsBecause public health policies can be influenced by results of mathematical modeling studies, it is important to conduct parameter identifiability analyses prior to fitting the models to available data and to report parameter estimates with quantified uncertainty. The method described is helpful in these regards and enhances the essential toolkit for conducting model-based inferences using compartmental dynamic models.Electronic supplementary materialThe online version of this article (10.1186/s12976-018-0097-6) contains supplementary material, which is available to authorized users.

Real-time forecasts of the COVID-19 epidemic in China from February 5th to February 24th, 2020

“…We then use a parametric bootstrap approach to quantify uncertainty around the best-fit solution, assuming a Poisson error structure. A detailed description of this method is provided in prior studies (Chowell, 2017;Roosa & Chowell, 2019). The models are refitted to the M ¼ 200 bootstrap datasets to obtain M parameter sets, which are used to define 95% confidence intervals for each parameter.…”

“…However, human-to-human transmission has driven its rapid spread with a total of 37,289 confirmed cases, including 813 deaths, in China and 302 confirmed cases imported in multiple countries as of February 9, 2020 (Chinese National Health Committee). While the early transmission potential of this novel coronavirus appeared similar to that of severe acute respiratory syndrome (SARS) (Riou & Althaus, 2020), the current tally of the epidemic has already surpassed the total cases reported for the SARS outbreaks in 2002e2003 (W. World Health Organization, 2003Wu, Leung, & Leung, 2020;Zhang et al, 2020).…”

A novel sub-epidemic modeling framework for short-term forecasting epidemic waves

“…Lack of identifiability or non-identifiability arises when one or more parameter estimates are associated with large uncertainties. This may be attributed to the model structure (structural identifiability) or due to the lack of information in a given dataset, which could be associated with the number of observations and the spatial-temporal resolution of the data [23,24]. Because the time series of incident cases in the observed epidemic wave is an aggregation of the overlapping sub-epidemics, different sub-epidemic profiles may give rise to indistinguishable aggregated epidemic waves.…”

Short-term Forecasts of the COVID-19 Epidemic in Guangdong and Zhejiang, China: February 13–23, 2020

“…Figures 1 and 2 contain the estimated ranges of cumulative case counts from 5-and 10-day forecasts for Guangdong and Zhejiang, respectively. 10-day ahead forecasts from each model with the reported calibration data are shown in references [9,14]. We refit the models to each of the M = 200 datasets generated by the bootstrap approach, resulting in M best-fit parameter sets that are used to construct the 95% confidence intervals for each parameter.…”

“…We use a parametric bootstrap approach to generate uncertainty bounds around the best-fit solution assuming a Poisson error structure; detailed descriptions of this method are provided in references [9,14]. We refit the models to each of the M = 200 datasets generated by the bootstrap approach, resulting in M best-fit parameter sets that are used to construct the 95% confidence intervals for each parameter.…”