National and Regional Influenza-Like-Illness Forecasts for the USA

Abstract

Health planners use forecasts of key metrics associated with influenza-like-illness (ILI); near-term weekly incidence, week of season onset, week of peak, and intensity of peak. Here, we describe our participation in a weekly prospective ILI forecasting challenge for the United States for the 2016-17 season and subsequent evaluation of our performance. We implemented a metapopulation model framework with 32 model variants. Variants differed from each other in their assumptions about: the force-of-infection (FOI); use of uninformative priors; the use of discounted historical data for not-yet-observed time points; and the treatment of regions as either independent or coupled. Individual model variants were chosen subjectively as the basis for our weekly forecasts; however, a subset of coupled models were only available part way through the season. Most frequently, during the 2016-17 season, we chose; FOI variants with both school vacations and humidity terms; uninformative priors; the inclusion of discounted historical data for not-yet-observed time points; and coupled regions (when available). Our near-term weekly forecasts substantially over-estimated incidence early in the season when coupled models were not available. However, our forecast accuracy improved in absolute terms and relative to other teams once coupled solutions were available. In retrospective analysis, we found that the 2016-17 season was not typical: on average, coupled models performed better when fit without historically augmented data. Also, we tested a simple ensemble model for the 2016-17 season and found that it underperformed our subjective choice for all forecast targets. In this study, we were able to improve accuracy during a prospective forecasting exercise by coupling dynamics between regions. Although reduction of forecast subjectivity should be a long-term goal, some degree of human intervention is likely to improve forecast accuracy in the medium-term in parallel with the systematic consideration of more sophisticated ensemble approaches.It is estimated that there are between 3 and 5 million worldwide annual seasonal cases 1 of severe influenza illness, and between 290 000 and 650 000 respiratory deaths [1].2 Influenza-like-illness (ILI) describes a set of symptoms and is a practical way for 3 health-care workers to easily estimate likely influenza cases. The Centers for Disease 4 Control (CDC) collects and disseminates ILI information, and has, for the last several 5 years, run a forecasting challenge (the CDC Flu Challenge) for modelers to predict 6 near-term weekly incidence, week of season onset, week of peak, and intensity of peak. 7 We have developed a modeling framework that accounts for a range of mechanisms 8 thought to be important for influenza transmission, such as climatic conditions, school 9 vacations, and coupling between different regions. In this study we describe our forecast 10 procedure for the 2016-17 season and highlight which features of our models resulted in 11 better or worse forecasts. Most notably, we ...

SARIMA Forecasts of Dengue Incidence in Brazil, Mexico, Singapore, Sri Lanka, and Thailand: Model Performance and the Significance of Reporting Delays

“…Thus, an obvious next step is to use a similar assessment framework to test forecasts made with dynamic models, which, ultimately should provide better forecasting capabilities, whilst at the same time, providing clues about the relevant processes. We anticipate that while statistical models may outperform mechanistic approaches on short horizons ( < 4 months, say), mechanistic models will likely provide better predictions at longer time horizons [24].…”

“…The retrospective analysis performed here did not account for any reporting errors; however, data published in near-real-time would be subject to a range of errors, including under-reporting, mis-diagnosis of symptoms, and/or false test results. Later, these would presumably be adjusted as the sources of error are corrected (e.g., [24]).…”

“…The copyright holder for this preprint this version posted June 29, 2020. ; https://doi.org/10.1101/2020.06.26.20141093 doi: medRxiv preprint outperform mechanistic approaches on short horizons (< 4 months, say), mechanistic models will likely provide better predictions at longer time horizons [24].…”

Accounting for Healthcare-Seeking Behaviours and Testing Practices in Real-Time Influenza Forecasts

“…Seasonal influenza is an archetypal example, which routinely presents a substantial challenge to healthcare systems in temperate climates, and whose clinical severity profile can differ substantially from one year to the next, and from setting to setting (e.g., city to city). Infectious disease forecasting is now beginning to establish itself as a useful decision-support tool for public health preparedness and response activities, particularly for seasonal influenza [2,3,4,5,6,7,8,9,10,14,15]. One of the many challenges for this field is to account for how behaviour changes affect surveillance data, and how these data should be interpreted [7,8,9,10].…”

“…For example, there is a well-established need to “understand the processes that determine how persons are identified by surveillance systems in order to appropriately adjust for the biases that may be present” [1]. The importance of understanding how people are identified by a surveillance system is particularly problematic in the context of near-real-time influenza forecasting [2,3,4,5,6,7,8,9,10], because healthcare-seeking behaviours and clinical decision-making are dynamic [11,12] and are subject to acute influences (e.g., media coverage [13]). The challenge that this poses is further compounded by delays in data collection and reporting, which reduce forecast performance [9,14,15].…”