A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave

Abbott, Sam; Burgard, Jan Pablo; Funk, Sebastian; Li, Michael Lingzhi; Bertsimas, Dimitris; Bouardi, Hamza Tazi; Lami, Omar Skali; Soni, Saksham; Abbott, Sam; Bosse, Nikos I.; Funk, Sebastian; Barbarossa, Maria Vittoria; Fuhrmann, Jan; Meinke, Jan H.; Bracher, Johannes; Deuschel, Jannik; Gneiting, Tilmann; Görgen, Konstantin; Ketterer, Jakob; Schienle, Melanie; Ullrich, Alexander; Wolffram, Daniel; Górski, Łukasz; Gruziel-Słomka, Magdalena; Kaczorek, Artur; Moszyński, Antoni; Niedzielewski, Karol; Nowosielski, Jędrzej; Radwan, Maciej; Rakowski, Franciszek; Semeniuk, Marcin; Zieliński, Jakub; Bartczuk, Rafał P.; Kisielewski, Jan; Bhatia, Sangeeta; Biecek, Przemysław; Bezborodov, Viktor; Krueger, Tyll; Burgard, Jan Pablo; Heyder, Stefan; Hotz, Thomas; Osthus, Dave; Michaud, Isaac; Castro, Lauren; Fairchild, Geoffrey; Kheifetz, Yuri; Kirsten, Holger; Scholz, Markus; Gambin, Anna; Gogolewski, Krzysztof; Miasojedow, Błażej; Rabczenko, Daniel; Rosińska, Magdalena; Bawiec, Marek A.; Bodych, Marcin; Ożański, Tomasz; Pabjan, Barbara; Rafajłlowicz, Ewaryst; Skubalska-Rafajłowicz, Ewa; Rafajłowicz, Ewaryst; Migalska, Agata; Szczurek, Ewa; Flahault, Antoine; Manetti, Elisa; Choirat, Christine; Haro, Benjamín Béjar; Krymova, Ekaterina; Lee, Gavin; Obozinski, Guillaume; Sun, Tao; Thanou, Dorina; Gu, Quanquan; Xu, Pan; Chen, Jinghui; Wang, Lingxiao; Zou, Difan; Zhang, Weitong; Srivastava, Ajitesh; Prasanna, Viktor K.; Xu, Frost Tianjian

doi:10.1038/s41467-021-25207-0

Cited by 59 publications

(77 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found that the mean-ensemble model made the most accurate (as measured by median rWIS) and most consistently accurate (as measured by rWIS IQR) forecasts across forecast horizons, forecast dates and Trusts, overcoming the variable performance of the individual models. This is consistent with other COVID-19 forecast evaluation studies [ 8 , 14 , 21 , 23 ] and other diseases [ 25 , 27 , 48 ].…”

Section: Discussionsupporting

confidence: 92%

“…However, the utility of cases as a predictor for admissions is limited by the quality of case forecasts: whilst perfect case forecasts can improve forecasts of admissions, real-time case forecasts are not perfect and can lead to worse forecasts of admissions than simple trend-based models. Unfortunately, making accurate forecasts of COVID-19 cases in a rapidly-evolving epidemic is challenging [ 23 , 47 ], especially in the face of changing local restrictions. The Rt-based case forecasting model used here assumes no change in future Rt, so cannot anticipate sudden changes in transmission, for example due to a change in policy such as lockdowns.…”

Section: Discussionmentioning

confidence: 99%

“…This reduces reliance on a single forecasting model and, given a minimum quality of the constituent models, the average performance of ensembles is generally comparable, if not better than, its best constituent models [ 8 , 21 ]. Ensemble methods have been widely used in real-time during the COVID-19 pandemic to leverage the contributions of multiple modelling groups to a single forecasting task [ 8 , 22 , 23 ], as well as previously during outbreaks of influenza [ 18 , 24 ], Ebola virus disease [ 25 ], dengue [ 26 ], and Zika [ 27 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level

Meakin

Abbott

Bosse

et al. 2022

BMC Med

Self Cite

View full text Add to dashboard Cite

Background Forecasting healthcare demand is essential in epidemic settings, both to inform situational awareness and facilitate resource planning. Ideally, forecasts should be robust across time and locations. During the COVID-19 pandemic in England, it is an ongoing concern that demand for hospital care for COVID-19 patients in England will exceed available resources. Methods We made weekly forecasts of daily COVID-19 hospital admissions for National Health Service (NHS) Trusts in England between August 2020 and April 2021 using three disease-agnostic forecasting models: a mean ensemble of autoregressive time series models, a linear regression model with 7-day-lagged local cases as a predictor, and a scaled convolution of local cases and a delay distribution. We compared their point and probabilistic accuracy to a mean-ensemble of them all and to a simple baseline model of no change from the last day of admissions. We measured predictive performance using the weighted interval score (WIS) and considered how this changed in different scenarios (the length of the predictive horizon, the date on which the forecast was made, and by location), as well as how much admissions forecasts improved when future cases were known. Results All models outperformed the baseline in the majority of scenarios. Forecasting accuracy varied by forecast date and location, depending on the trajectory of the outbreak, and all individual models had instances where they were the top- or bottom-ranked model. Forecasts produced by the mean-ensemble were both the most accurate and most consistently accurate forecasts amongst all the models considered. Forecasting accuracy was improved when using future observed, rather than forecast, cases, especially at longer forecast horizons. Conclusions Assuming no change in current admissions is rarely better than including at least a trend. Using confirmed COVID-19 cases as a predictor can improve admissions forecasts in some scenarios, but this is variable and depends on the ability to make consistently good case forecasts. However, ensemble forecasts can make forecasts that make consistently more accurate forecasts across time and locations. Given minimal requirements on data and computation, our admissions forecasting ensemble could be used to anticipate healthcare needs in future epidemic or pandemic settings.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level

Meakin

Abbott

Bosse

et al. 2022

BMC Med

Self Cite

View full text Add to dashboard Cite

show abstract

“…In response, several organized modeling efforts were started to give public health officials as up to date information as possible about the trajectory of COVID-19 in the US and in Europe [11][12][13][14].…”

Section: Introductionmentioning

confidence: 99%

“…In addition to the US COVID-19 forecast hub, there are COVID-19 hubs that collect computational forecasts for Europe and specifically for Germany and Poland [11,13,14]. The majority of models submitted to these hubs are computational: statistical or dynamical models trained on structured data.…”

Section: Introductionmentioning

confidence: 99%

Chimeric forecasting: combining probabilistic predictions from computational models and human judgment

McAndrew¹,

Codi²,

Cambeiro³

et al. 2022

Preprint

View full text Add to dashboard Cite

Forecasts of the trajectory of an infectious agent can help guide public health decision making. A traditional approach to forecasting fits a computational model to structured data and generates a predictive distribution. However, human judgment has access to the same data as computational models plus experience, intuition, and subjective data. We propose a chimeric ensemble-a combination of computational and human judgment forecasts-as a novel approach to predicting the trajectory of an infectious agent. Each month from January, 2021 to June, 2021 we asked two generalist crowds, using the same criteria as the COVID-19 Forecast Hub, to submit a predictive distribution over incident cases and deaths at the US national level either two or three weeks into the future and combined these human judgment forecasts with forecasts from computational models submitted to the COVID-19 Forecasthub into a chimeric ensemble. We find a chimeric ensemble compared to an ensemble including only computational models improves predictions of incident cases and shows similar performance for predictions of incident deaths. A chimeric ensemble is a flexible, supportive public health tool and shows promising results for predictions of the spread of an infectious agent.

show abstract

An Overview and General Framework for Spatiotemporal Modeling and Applications in Transportation and Public Health

Tsui

Zhao

2022

Artificial Intelligence, Big Data and Data Science in Statistics

View full text Add to dashboard Cite

A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave

Cited by 59 publications

References 42 publications

Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level

Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level

Chimeric forecasting: combining probabilistic predictions from computational models and human judgment

An Overview and General Framework for Spatiotemporal Modeling and Applications in Transportation and Public Health

Contact Info

Product

Resources

About