Robustness and bias of European excess death estimates in 2020 under varying model specifications

Abstract

Various procedures are in use to calculate excess deaths during the ongoing COVID-19 pandemic. Using weekly death counts from 20 European countries, we evaluate the robustness of excess death estimates to the choice of model for expected deaths and perform a cross-validation analysis to assess the error and bias in each model's predicted death counts. We find that the different models produce very similar patterns of weekly excess deaths but disagree substantially on the level of excess. While the exact country ranking along percent excess death in 2020 is sensitive to the choice of model the top and bottom ranks are robustly identified. On the country level, the 5-year average death rate model tends to produce the lowest excess death estimates, whereas high excess deaths are produced by the popular 5-year average death count and Euromomo-style Serfling models. Cross-validation revealed these estimates to be biased under a causal interpretation of "expected deaths had COVID-19 not happened."

Global and National Declines in Life Expectancy: An End‐of‐2021 Assessment

“…Estimates of declines in life expectancy were derived here from numbers of excess deaths that in turn must be derived from statistical modeling of what the number of deaths might have been in the absence of the pandemic. Even in countries with the required good-quality data, this modeling involves multiple decisions for which there is no clear rule-regarding the number of past years used to define benchmark mortality conditions, if and how a temporal mortality trend is modeled, and so forth-and which may substantially impact the results (Nepomuceno et al 2021;Schöley 2021). The main challenge to measuring excess deaths with confidence, however, remains substantial data limitations in many parts of the world.…”

Assessing excess mortality in Vienna and Austria after the first year of the COVID-19 pandemic

“…To compare the impact of COVID-19 on mortality in Vienna with that in other metropolises, it would be necessary to look beyond Austria. Regional comparative analyses at the European level (Goujon et al, 2021;Schöley, 2021;EuroMOMO, 2021) are still limited, since Eurostat does not (yet) provide up-to-date mortality data for European cities and regions with a level of demographic detail similar to the level we used in our analysis (based on data provided by Statistik Austria). With respect to further research, we are currently investigating the availability of such data from European cities, which would allow us to analyse the specific characteristics of COVID-19-related excess mortality in the urban context.…”

“…By definition, excess mortality is the number of deaths from all causes in a given period relative to the number of deaths that would have normally been expected to occur in that period. For assessing the direct and indirect impacts of COVID-19 on overall mortality, excess mortality for all causes of deaths is a well-established indicator (Aburto et al, 2021;Aron et al, 2020;Ghislandi et al, 2021;Schöley, 2021).…”

Life expectancy drop in 2020. Estimates based on Human Mortality Database

“…Measuring the effects of COVID-19 pandemic in terms of life expectancy drop might be useful in a comparative perspective, as other measures are flawed by issues that make comparisons more difficult: COVID-19 deaths are limited by missclassification problems [ 16 ], while excess mortality (i.e. the difference between the number of observed deaths in 2020 and the expected number that would have been observed without the pandemic) may be extremely variable, depending on the way the baseline is estimated (see Schöley [ 17 ] and Nepomuceno et al . [ 18 ]).…”

“…Measuring the effects of COVID-19 pandemic in terms of life expectancy drop might be useful in a comparative perspective, as other measures are flawed by issues that make comparisons more difficult: COVID-19 deaths are limited by missclassification problems [16], while excess mortality (i.e. the difference between the number of observed deaths in 2020 and the expected number that would have been observed without the pandemic) may be extremely variable, depending on the way the baseline is estimated (see Scho ¨ley [17] and Nepomuceno et al [18]). One issue of life expectancy at birth is that it is often misintepreted, due to the fact it refers to an hypothetical cohort and not a real one, hence the best way to understand what the figures in Table 1 really mean, is to compare them with the life expectancy drops or gains in past years.…”