Deviations in Predicted COVID-19 cases in the US during early months of 2021 relate to rise in B.1.526 and its family of variants

Nandakishore, Pawan; Lui, Meng; Prakash, Rishab; Gourneni, Surya; Sukumaran, Rohan; Berman, Jonathan M.; Iyer, Swaminathan P.; Glorioso, Christin

doi:10.1101/2021.12.06.21267388

Cited by 2 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the accuracy of a compartment model depends heavily on accurate estimates of the R 0 in a population, a variable that changes over time, especially as new virus variants emerge. For example, the emergence of the Iota variant of SARS-CoV-2 (also known as lineage B.1.526) resulted in an unpredicted increase in case counts ( 49 ). Machine learning models train algorithms that would be difficult to develop by conventional means.…”

Section: Discussionmentioning

confidence: 99%

Accuracy of US CDC COVID-19 forecasting models

Chharia,

Jeevan,

Jha

et al. 2024

Front. Public Health

View full text Add to dashboard Cite

Accurate predictive modeling of pandemics is essential for optimally distributing biomedical resources and setting policy. Dozens of case prediction models have been proposed but their accuracy over time and by model type remains unclear. In this study, we systematically analyze all US CDC COVID-19 forecasting models, by first categorizing them and then calculating their mean absolute percent error, both wave-wise and on the complete timeline. We compare their estimates to government-reported case numbers, one another, as well as two baseline models wherein case counts remain static or follow a simple linear trend. The comparison reveals that around two-thirds of models fail to outperform a simple static case baseline and one-third fail to outperform a simple linear trend forecast. A wave-by-wave comparison of models revealed that no overall modeling approach was superior to others, including ensemble models and errors in modeling have increased over time during the pandemic. This study raises concerns about hosting these models on official public platforms of health organizations including the US CDC which risks giving them an official imprimatur and when utilized to formulate policy. By offering a universal evaluation method for pandemic forecasting models, we expect this study to serve as the starting point for the development of more accurate models.

show abstract

Section: Discussionmentioning

confidence: 99%

Accuracy of US CDC COVID-19 forecasting models

Chharia,

Jeevan,

Jha

et al. 2024

Front. Public Health

View full text Add to dashboard Cite

show abstract

“…However, the accuracy of a compartment model depends heavily on accurate estimates of the R 0 in a population, a variable that changes over time, especially as new virus variants emerge. For example, the emergence of the Iota variant of SARS-CoV-2 (also known as lineage B.1.526) resulted in an unpredicted increase in case counts (11). Machine learning models train algorithms that would be difficult to develop by conventional means.…”

Section: Discussionmentioning

confidence: 99%

Accuracy of US CDC COVID-19 Forecasting Models

Chharia

Jeevan

Jha

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate predictive modeling of pandemics is essential for optimally distributing resources and setting policy. Dozens of case predictions models have been proposed but their accuracy over time and by model type remains unclear. In this study, we analyze all US CDC COVID-19 forecasting models, by first categorizing them and then calculating their mean absolute percent error, both wave-wise and on the complete timeline. We compare their estimates to government-reported case numbers, one another, as well as two baseline models wherein case counts remain static or follow a simple linear trend. The comparison reveals that more than one-third of models fail to outperform a simple static case baseline and two-thirds fail to outperform a simple linear trend forecast. A wave-by-wave comparison of models revealed that no overall modeling approach was superior to others, including ensemble models, and error in modeling has increased over time during the pandemic. This study raises concerns about hosting these models on official public platforms of health organizations including the US-CDC which risks giving them an official imprimatur and further raising concerns if utilized to formulate policy. By offering a universal evaluation method for pandemic forecasting models, we expect this work to serve as the starting point towards the development of more sophisticated models.

show abstract

Deviations in Predicted COVID-19 cases in the US during early months of 2021 relate to rise in B.1.526 and its family of variants

Cited by 2 publications

References 16 publications

Accuracy of US CDC COVID-19 forecasting models

Accuracy of US CDC COVID-19 forecasting models

Accuracy of US CDC COVID-19 Forecasting Models

Contact Info

Product

Resources

About