A Quantitative Evaluation of COVID-19 Epidemiological Models

Yogurtcu, Osman N.; Messan, Marisabel Rodriguez; Gerkin, Richard C.; Belov, Artur; Yang, H; Ra, Forshee; Chow, C. C.

doi:10.1101/2021.02.06.21251276

Cited by 2 publications

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Accuracy of US CDC COVID-19 Forecasting Models

Chharia

Jeevan

Jha

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate predictive modeling of pandemics is essential for optimally distributing resources and setting policy. Dozens of case predictions models have been proposed but their accuracy over time and by model type remains unclear. In this study, we analyze all US CDC COVID-19 forecasting models, by first categorizing them and then calculating their mean absolute percent error, both wave-wise and on the complete timeline. We compare their estimates to government-reported case numbers, one another, as well as two baseline models wherein case counts remain static or follow a simple linear trend. The comparison reveals that more than one-third of models fail to outperform a simple static case baseline and two-thirds fail to outperform a simple linear trend forecast. A wave-by-wave comparison of models revealed that no overall modeling approach was superior to others, including ensemble models, and error in modeling has increased over time during the pandemic. This study raises concerns about hosting these models on official public platforms of health organizations including the US-CDC which risks giving them an official imprimatur and further raising concerns if utilized to formulate policy. By offering a universal evaluation method for pandemic forecasting models, we expect this work to serve as the starting point towards the development of more sophisticated models.

show abstract

Accuracy of US CDC COVID-19 Forecasting Models

Chharia

Jeevan

Jha

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Accuracy of US CDC COVID-19 forecasting models

Chharia,

Jeevan,

Jha

et al. 2024

Front. Public Health

View full text Add to dashboard Cite

Accurate predictive modeling of pandemics is essential for optimally distributing biomedical resources and setting policy. Dozens of case prediction models have been proposed but their accuracy over time and by model type remains unclear. In this study, we systematically analyze all US CDC COVID-19 forecasting models, by first categorizing them and then calculating their mean absolute percent error, both wave-wise and on the complete timeline. We compare their estimates to government-reported case numbers, one another, as well as two baseline models wherein case counts remain static or follow a simple linear trend. The comparison reveals that around two-thirds of models fail to outperform a simple static case baseline and one-third fail to outperform a simple linear trend forecast. A wave-by-wave comparison of models revealed that no overall modeling approach was superior to others, including ensemble models and errors in modeling have increased over time during the pandemic. This study raises concerns about hosting these models on official public platforms of health organizations including the US CDC which risks giving them an official imprimatur and when utilized to formulate policy. By offering a universal evaluation method for pandemic forecasting models, we expect this study to serve as the starting point for the development of more accurate models.

show abstract

A Quantitative Evaluation of COVID-19 Epidemiological Models

Cited by 2 publications

References 11 publications

Accuracy of US CDC COVID-19 Forecasting Models

Accuracy of US CDC COVID-19 Forecasting Models

Accuracy of US CDC COVID-19 forecasting models

Contact Info

Product

Resources

About