Refining Epidemiological Forecasts with Simple Scoring Rules

Abstract

Estimates from infectious disease models have constituted a significant part of the scientific evidence used to inform the response to the COVID-19 pandemic in the UK. These estimates can vary strikingly in their precision, with some being over-confident and others over-cautious. The uncertainty in epidemiological forecasts should be commensurate with the errors in their predictions. We propose Normalised Estimation Error Squared (NEES) as a metric for assessing the consistency between forecasts and future observations. We introduce a novel infectious disease model for COVID-19 and use it to demonstrate the usefulness of NEES for diagnosing over-confident and over-cautious predictions resulting from different values of a regularization parameter.

Inference of Stochastic Disease Transmission Models Using Particle-MCMC and a Gradient Based Proposal

“…Table I gives a description of the parameters used in the statistical models below. Prior information for these parameters is also included and taken from [15].…”

“…In this piece of work we have only considered simulated data to provide a proof of concept. A next step would be to apply these methods to more sophisticated epidemiological models, as seen in [15], and to use real data pertinent to disease outbreaks.…”

“…Hamiltonian Monte Carlo (HMC) [11] and the No-U-Turn Sampler (NUTS) [12] use gradient information about the logposterior of θ so can efficiently explore π(θ). Probabilistic programming languages (ppls), such as Stan [13], have made MCMC an accessible tool for calibrating epidemiological models [14], [15] because these ppls use NUTS.…”