Enhancing long-term forecasting: Learning from COVID-19 models

Abstract

While much effort has gone into building predictive models of the COVID-19 pandemic, some have argued that early exponential growth combined with the stochastic nature of epidemics make the long-term prediction of contagion trajectories impossible. We conduct two complementary studies to assess model features supporting better long-term predictions. First, we leverage the diverse models contributing to the CDC repository of COVID-19 USA death projections to identify factors associated with prediction accuracy across different projection horizons. We find that better long-term predictions correlate with: (1) capturing the physics of transmission (instead of using black-box models); (2) projecting human behavioral reactions to an evolving pandemic; and (3) resetting state variables to account for randomness not captured in the model before starting projection. Second, we introduce a very simple model, SEIRb, that incorporates these features, and few other nuances, offers informative predictions for as far as 20-weeks ahead, with accuracy comparable with the best models in the CDC set. Key to the long-term predictive power of multi-wave COVID-19 trajectories is capturing behavioral responses endogenously: balancing feedbacks where the perceived risk of death continuously changes transmission rates through the adoption and relaxation of various Non-Pharmaceutical Interventions (NPIs).

Quantifying the COVID‐19 endgame: Is a new normal within reach?

“…We estimated the model using a wide range of data, including historical cases and deaths across 93 nations while accounting for vaccination, variants, changes in disease acuity, and the incidence of asymptomatic cases arising from vaccines, variants, and prior infection, and the impact of weather and cross‐national differences in demographics and hospital capacity. Importantly, and in contrast to other models (surveyed in Rahmandad et al ., 2022 ), the model includes endogenous allocation of testing and treatment capacity based on disease acuity, improvements in treatments, and especially endogenous behavioral responses to risk and the impact of pandemic fatigue. The model provides a framework to project future scenarios consistently with reasonable estimates for the many interdependent factors that influence what a new normal may look like in each country.…”

“…Prior work (e.g. Rahmandad et al ., 2021 , 2022 ) shows that these behavioral feedbacks are essential in explaining the multiple waves seen in the pandemic, orders‐of‐magnitude differences in death rates across nations, and for providing more reliable long‐term forecasts. This study identifies the two distinct regimes a community may face depending on the strength of the behavioral response feedback, provides estimates of where each nation may stand in this continuum, and projects the likely future of the pandemic.…”

“…We then build on recent research suggesting that long‐term projections for the pandemic are feasible and require mechanistic (causal) models in which community responses (e.g. adopting NPIs) respond endogenously to the evolution of the disease (Rahmandad et al ., 2022 ). Specifically, we expand the model described in Rahmandad et al .…”

“…The basic solution to this general problem is to “reset” the state variables based on observed data to correct for the impact of unobserved process noise and other factors excluded from the model (Eberlein, 2015 ). Prior research shows that forecasting performance across a wide range of COVID‐19 models is improved when state‐resetting is included (Rahmandad et al ., 2022 ). Formal state resetting methods such as Kalman filtering and particle filtering (Cazelles and Chau, 1997 ) recognize that noise in a system can cause the state variables to drift away from the best empirical values for them even if the model is correctly specified (for an intuitive example, see Forrester, 1961 , Appendix K).…”

Effect of mandating vaccination on COVID-19 cases in colleges and universities

“…Obtaining individual-level data and a larger sample of universities, gathering test frequency data before and after the intervention, and tracking cases during summer semesters to improve cross-university comparison are some of the potential extensions. Future studies should also examine several related behavioral factors that influence individuals’ change in risk perception and willingness to vaccinate ( Rahmandad et al, 2022 ). Common methodological assumptions such as consistent trends across universities, while consistent with our intuitions, are still strong assumptions, and given the lack of data on summer semesters are hard to fully confirm.…”