Can Auxiliary Indicators Improve COVID-19 Forecasting and Hotspot Prediction?

Abstract

Reliable, short-term forecasts of traditional public health reporting streams (such as cases, hospitalizations, and deaths) are a key ingredient in effective public health decision-making during a pandemic. Since April 2020, our research group has worked with data partners to collect, curate, and make publicly available numerous real-time COVID-19 indicators, providing multiple views of pandemic activity. This paper studies the utility of these indicators from a forecasting perspective. We focus on five indicators, derived from medical insurance claims data, web search queries, and online survey responses. For each indicator, we ask whether its inclusion in a simple model leads to improved predictive accuracy relative to a similar model excluding it. We consider both probabilistic forecasting of confirmed COVID-19 case rates and binary prediction of case “hotspots”. Since the values of indicators (and case rates) are commonly revised over time, we take special care to ensure that the data provided to a forecaster is the version that would have been available at the time the forecast was made. Our analysis shows that consistent but modest gains in predictive accuracy are obtained by using these indicators, and furthermore, these gains are related to periods in which the auxiliary indicators behave as “leading indicators” of case rates.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

“…An observational study could be conducted with forecasts collected by the Hub, but any such analysis would be confounded by other factors about how the model was built and validated. Other research in this area has shown small but measurable improvements in predictive accuracy by including other data streams available in real time ( 23 ). Continued research is needed to evaluate how behavioral, mobility, variant prevalence, or other data streams might enhance predictive modeling…”

Real-Time Estimation of COVID-19 Infections: Deconvolution and Sensor Fusion

“…The indicators enumerated in Section 2 have displayed impressive correlations to reported COVID-19 cases (Reinhart et al, 2021), and moreover, demonstrated an ability to improve the accuracy of case forecasting and hotspot prediction models (McDonald et al, 2021). In this section, we describe how to use each indicator to build a real-time sensor that estimates the latent infection rate, and how to fuse such estimates together into a single nowcast.…”

“…For more information on CTIS in particular, we refer to Salomon et al (2021). For a study of how these and similar indicators can contribute to COVID-19 forecasting models, we refer to McDonald et al (2021).…”

“…This is possible becausse the COVIDcast API records and provides access to all historical versions of data, as described in Reinhart et al (2021). As epidemic data is often subject to revision, if we train and evaluate models on "finalized" data (that would have been available only at a much later time point) then this can lead to overly optimistic (inaccurate) conclusions about real-time model performance; see, e.g., McDonald et al (2021).…”

COVIDNearTerm: A simple method to forecast COVID-19 hospitalizations

“…Another group of models uses geographic and aggregate mobility data and mechanistic transmission models such as SEIR to predict population incidence and hospitalization rates [17,18,19]. In terms of short-term predictions, two recent novel approaches include utilizing small area data together with empirical models to make short-term predictions [20] and using auxiliary indicators [21].…”