No abstract

Infectious disease forecasting has been a useful tool for public health planning and messaging during the COVID-19 pandemic. In partnership with the CDC, the organizers of the COVID-19 Forecast Hub have created a mechanism for forecasters from academia, industry, and government organizations to submit weekly near-term predictions of COVID-19 targets in the United States. Here we describe our efforts to participate in the COVID-19 Forecast Hub through the Forecasting COVID-19 in the United States (FOCUS) project. The effort led to more than three months of weekly submissions and development of an automated pipeline to generate forecasts. The models used in FOCUS yielded forecasts that ranked relatively well in terms of precision and accuracy.

Near-term probabilistic forecasts for infectious diseases such as COVID-19 and influenza play an important role in public health communication and policymaking. From 2013-2019, the FluSight challenge run by the Centers for Disease Control and Prevention invited researchers to develop and submit forecasts using influenza-like illness (ILI) as a measure of influenza burden. Here we examine how several statistical models and an autoregressive neural network model perform for forecasting ILI during the COVID-19 pandemic, where historical patterns of ILI were highly disrupted. We find that the autoregressive neural network model which forecasted ILI well pre-COVID still performs well for some locations and forecast horizons, but its performance is highly variable, and performs poorly in many cases. We found that a simple exponential smoothing statistical model is in the top half of ranked models we evaluated nearly 75% of the time. Our results suggest that even simple statistical models may perform as well as or better than more complex machine learning models for forecasting ILI during the COVID-19 pandemic. We also created an ensemble model from the limited set of time series forecast models we created here. The limited ensemble model was rarely the best or the worst performing model compared to the rest of the models assessed, confirming previous observations from other infectious disease forecasting efforts on the less variable and generally favorable performance of ensemble forecasts. Our results support previous findings that no single modeling approach outperforms all other models across all locations, time points, and forecast horizons, and that ensemble forecasting consortia such as the COVID-19 Forecast Hub and FluSight continue to serve valuable roles in collecting, aggregating, and ensembling forecasts using fundamentally disparate modeling strategies.