Significance This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the United States. Results show high variation in accuracy between and within stand-alone models and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public-health action.

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance Statement This paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

As the spread of COVID19 in the US continues to grow, local and state officials face difficult decisions about when and how to transition to a "new normal." The goal of this study is to project the number of COVID19 infections and resulting severe outcomes, and the need for hospital capacity under social distancing, particularly, shelter-in-place and voluntary quarantine for the State of Georgia. We developed an agent-based simulation model to project the infection spread. The model utilizes COVID19-specific parameters and data from Georgia on population interactions and demographics. The simulation study covered a seven and a half-month period, testing different social distancing scenarios, including baselines (nointervention or school closure only) and combinations of shelter-in-place and voluntary quarantine with different timelines and compliance levels. The following outcomes are compared at the state and community levels: the number and percentage of cumulative and daily new symptomatic and asymptomatic infections, hospitalizations, and deaths; COVID19-related demand for hospital beds, ICU beds, and ventilators. The results suggest that shelter-inplace followed by voluntary quarantine reduced peak infections from approximately 180K under no intervention and 113K under school closure, respectively, to below 53K, and delayed the peak from April to July or later. Increasing shelter-in-place duration from four to five weeks yielded 2-9% and 3-11% decrease in cumulative infection and deaths, respectively. Regardless of the shelter-in-place duration, increasing voluntary quarantine compliance decreased daily new infections from almost 53K to 25K, and decreased cumulative infections by about 50%. The cumulative number of deaths ranged from 6,660 to 19,430 under different scenarios. Peak infection date varied across scenarios and counties; on average, increasing shelter-in-place duration delayed the peak day by 6 days. Overall, shelter-in-place followed by voluntary quarantine substantially reduced COVID19 infections, healthcare resource needs, and severe outcomes.

medRxiv preprint KEY POINTS Question: How social distancing strategies impact the spread of COVID19?Findings: Extending shelter-in-place by one week delays the peak by about 8 days but it does not significantly reduce the peak. High compliance with voluntary quarantine following shelter-in-place reduces the peak by 40% in Georgia.Meaning: There needs to be a very strong public messaging about social distancing when shelter-in-place is lifted, to achieve a better match between healthcare capacity and demand, considering different peak times across the communities.ABSTRACT Importance As the COVID19 spread in the US continues to grow, local and state officials face difficult decisions about when and how to transition to a "new normal."Objective Project the number of COVID19 infections and resulting severe outcomes, and the need for hospital capacity under social distancing, particularly, shelter-in-place and voluntary quarantine. DesignWe developed an agent-based simulation model to project the infection spread. We populated the model using COVID19-specific parameters for the natural history of the disease and data from Georgia on agents' interactions and demographics.Setting The simulation study covered a six-month period, testing different social distancing scenarios, including baselines (no-intervention or school closure only) and combinations of shelter-in-place and voluntary quarantine with different timelines and compliance levels. The outcomes are compared at the state and community levels. Main OutcomesThe number and percentage of cumulative and daily new and symptomatic and asymptomatic infections, hospitalizations, and deaths; COVID19related demand for hospital beds, ICU beds, and ventilators. ResultsThe combined intervention of shelter-in-place followed by voluntary quarantine reduced peak infections from 180,000 under no intervention and 120,000 under school closure, respectively, to below 80,000, and delayed the peak from April to June or later.Increasing shelter-in-place duration from four to five weeks yielded 3-14% and 4-6% decrease in cumulative infection and fatality rates, respectively. Regardless of the All rights reserved. No reuse allowed without permission.was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. : medRxiv preprint shelter-in-place duration, increasing voluntary quarantine compliance decreased daily new infections from almost 80,000 to 50,000, and decreased cumulative infection rate by 50%. The total number of fatalities ranged from 6,150 to 17,900 under different scenarios. Peak infection date varied across scenarios and counties; on average, increasing shelter-in-place duration delayed the peak day by 7 days across counties.The peak percentage is similar across rural and urban counties. Region D is estimated to have the highest COVID19-related healthcare needs with 7,357 hospital beds, 1,141 ICU beds, and 558 ventilators. Conclusions and RelevanceShelter-in-place followed by voluntary quarantine substanti...