Retrospective evaluation of short-term forecast performance of ensemble sub-epidemic frameworks and other time-series models: The 2022-2023 mpox outbreak across multiple geographical scales, July 14^th, 2022, through February 26th, 2023

Abstract

In May 2022, public health officials noted an unprecedented surge in Mpox cases in non-endemic countries worldwide. As the epidemic accelerated, multi-model forecasts of the epidemic's trajectory were critical in guiding the implementation of public health interventions, and in determining policy. As the case levels have significantly decreased as of mid-August, evaluating model performance is essential to advance the growing field of epidemic forecasting. Using laboratory-confirmed Mpox case data from the CDC and OWID teams through the week of January 26th, 2023, we generated retrospective sequential weekly forecasts (e.g., 1-week, 2-week, 3-week, and 4-week) for Brazil, Canada, England, France, Germany, Spain, the USA, and at the global scale using models that require minimal input data including the auto-regressive integrated moving average (ARIMA), general additive model (GAM), simple linear regression (SLR) as well as the spatial-wave and n-sub-epidemic modeling frameworks. Forecast performance was assessed using MSE, MAE, WIS, 95% PI coverage metrics as well as skill scores. Overall, the spatial-wave sub-epidemic modeling framework outcompeted other models across most locations and forecasting horizons in terms of the average MSE, MAE, and WIS. It was followed closely in success by the n-sub-epidemic top-ranked and weighted ensemble (2) models. Regarding average 95% PI coverage, the n-sub-epidemic unweighted (equally weighted top-models) ensemble (2) model performed best across all forecasting horizons for most locations. However, many locations had multiple models performing equally well in terms of the average 95% PI coverage. Both the n-sub-epidemic and spatial-wave frameworks improved considerably in average MSE, MAE, and WIS, and minimally (<10%) in average 95% PI coverage relative to the ARIMA model. Findings lend further support to sub-epidemic models for short-term forecasting epidemics of emerging and re-emerging infectious diseases.

Evaluating the forecasting performance of ensemble sub-epidemic frameworks and other time series models for the 2022–2023 mpox epidemic

“…Data accessibility. Data and relevant code for this research work are stored in GitHub [70] and have been archived within the Zenodo repository [71].…”