Forecasting seasonal influenza in the U.S.: A collaborative multi-year, multi-model assessment of forecast performance

Abstract

Influenza infects an estimated 9 to 35 million individuals each year in the United States and is a contributing cause for between 12,000 and 56,000 deaths annually. Seasonal outbreaks of influenza are common in temperate regions of the world, with highest incidence typically occurring in colder and drier months of the year. Real-time forecasts of influenza transmission can inform public health response to outbreaks. We present the results of a multi-institution collaborative effort to standardize the collection and evaluation of forecasting models for influenza in the US for the 2010/2011 through 2016/2017 influenza seasons. For these seven seasons, we assembled weekly real-time forecasts of 7 targets of public health interest from 22 different models. We compared forecast accuracy of each model relative to a historical baseline seasonal average. Across all regions of the US, over half of the models showed consistently better performance than the historical baseline when forecasting incidence of influenza-like illness 1, 2 and 3 weeks ahead of available data and when forecasting the timing and magnitude of the seasonal peak. In some regions, delays in data reporting were strongly and negatively associated with forecast accuracy. More timely reporting and an improved overall accessibility to novel and traditional data sources are needed to improve forecasting accuracy and its integration with real-time public health decision-making.

On the value of preprints: An early career researcher perspective

“…Expedited sharing of results in physiology and epidemiology as preprints can dramatically accelerate ECR research in interdisciplinary fields [24,28,31] as it has done in physical sciences [29,30]. In systems biology and systems medicine, preprints and open-access data can provide biochemical and physiological parameters that are key to development of complex multiscale computational models of human health and disease [31]. In the absence of open, diverse, and timely availability of research results, efficient use of such models that link molecular networks to cells, organs, and organ systems has been slow and challenging [32].…”

“…In the absence of open, diverse, and timely availability of research results, efficient use of such models that link molecular networks to cells, organs, and organ systems has been slow and challenging [32]. Accelerated release of biological results and methods for data integration will promptly inform evaluation of higher-resolution predictive computational models of human pathologies, boosting ECR research concerning personalized diagnostics and therapies [31,32].…”

On the value of preprints: an early career researcher perspective

“…Expedited sharing of results in physiology and epidemiology as preprints can dramatically accelerate ECR research in interdisciplinary fields as well (25) as it has done in physical sciences (26,27). In systems biology and systems medicine, preprints and open access data can provide biochemical, biophysical and physiological parameters which are key to development of complex multiscale computational models of human health and disease (28). In the absence of open, abundant, diverse and timely availability of research results, efficient use of computational models that link molecular networks to cells, organs and organ systems has been slow and challenging (29).…”

“…In the absence of open, abundant, diverse and timely availability of research results, efficient use of computational models that link molecular networks to cells, organs and organ systems has been slow and challenging (29). Accelerated release of biological results and methods for data integration will promptly inform evaluation of higher resolution predictive computational models of human pathologies, boosting ECR research concerning personalized diagnostics and therapies (28,29).…”

Accounting for healthcare-seeking behaviours and testing practices in real-time influenza forecasts

“…The importance of understanding how people are identified by a surveillance system is particularly problematic in the context of near-real-time influenza forecasting [2][3][4][5][6][7][8][9][10], because healthcare-seeking behaviours and clinical decision-making are dynamic [11,12] and are subject to acute influences (e.g., media coverage [13]). The challenge that this poses is further compounded by delays in data collection and reporting, which reduce forecast performance [9,14,15]. And in the event of a pandemic, changes in patient and clinician behaviours are likely to be even more pronounced, with ramifications not only for interpreting surveillance data [3], but also for delivering effective and proportionate public health responses [16].…”