Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic

Abstract

Aims. Markedly elevated adverse mental health symptoms were widely observed early in the coronavirus disease-2019 (COVID-19) pandemic. Unlike the U.S., where cross-sectional data indicate anxiety and depression symptoms have remained elevated, such symptoms reportedly declined in the U.K., according to analysis of repeated measures from a large-scale longitudinal study. However, nearly 40% of U.K. respondents (those who did not complete multiple follow-up surveys) were excluded from analysis, suggesting that survivorship bias might partially explain this discrepancy. We therefore sought to assess survivorship bias among participants in our longitudinal survey study as part of The COVID-19 Outbreak Public Evaluation (COPE) Initiative. Methods. Survivorship bias was assessed in 4039 U.S. respondents who completed surveys including the assessment of mental health as part of The COPE Initiative in April 2020 and were invited to complete follow-up surveys. Participants completed validated screening instruments for symptoms of anxiety, depression and insomnia. Survivorship bias was assessed for (1) demographic differences in follow-up survey participation, (2) differences in initial adverse mental health symptom prevalence adjusted for demographic factors and (3) differences in follow-up survey participation based on mental health experiences adjusted for demographic factors. Results. Adjusting for demographics, individuals who completed only one or two out of four surveys had significantly higher prevalence of anxiety and depression symptoms in April 2020 (e.g. one-survey v. four-survey, anxiety symptoms, adjusted prevalence ratio [aPR]: 1.30, 95% confidence interval [CI]: 1.08-1.55, p = 0.0045; depression symptoms, aPR: 1.43, 95% CI: 1.17-1.75, p = 0.00052). Moreover, individuals who experienced incident anxiety or depression symptoms had significantly higher adjusted odds of not completing follow-up surveys (adjusted odds ratio [aOR]: 1.68, 95% CI: 1.22-2.31, p = 0.0015, aOR: 1.56, 95% CI: 1.15-2.12, p = 0.0046, respectively). Conclusions. Our findings reveal significant survivorship bias among longitudinal survey respondents, indicating that restricting analytic samples to only respondents who provide repeated assessments in longitudinal survey studies could lead to overly optimistic interpretations of mental health trends over time. Cross-sectional or planned missing data designs may provide more accurate estimates of population-level adverse mental health symptom prevalence than longitudinal surveys.

Trends in Psychological Distress Among US Adults During Different Phases of the COVID-19 Pandemic

“…Limiting our sample to respondents to all 4 waves may have led to underestimation of the prevalence of serious distress owing to survivorship bias. 6 NORC AmeriSpeak uses probability-based recruitment to minimize sampling bias, and survey weights are adjusted for nonresponse, but the sample may not be representative of broader populations. 2 As the results of this study suggest, psychological distress among US adults has been elevated during the COVID-19 pandemic.…”

Assets, stressors, and symptoms of persistent depression over the first year of the COVID-19 pandemic

“…However, these studies were conducted during summer months, which may have been confounded by seasonal effects resulting in improved affect. In addition, a requirement that participants provide at least three repeated measures (i.e., participants had to respond to at least three of the weekly surveys between 23 March 2020 and 9 August 2020 to be included in the sample) in the study by Fancourt et al may be susceptible to survivorship bias ( 27 ), which could lead to an underreporting of adverse mental health symptoms at the population level. Given that the CLIMB survey had a response rate of 81.1% at T2, our data may be less susceptible to survivorship bias.…”

Design, content, and fieldwork procedures of the COVID‐19 Psychological Research Consortium (C19PRC) Study – Wave 4

“…For example, between April and September 2020, the US COVID‐19 Outbreak Public Evaluation (COPE) Initiative, a large, online longitudinal study ( N = 6548, baseline sample) which assesses public attitudes, behaviours, and beliefs relating to the pandemic, and evaluates mental and behavioural health during the pandemic, interviewed participants four times. Participation declined from 51.9% at the first follow‐up to 28.5% at the third follow‐up survey (Czeisler et al., 2021b ). Moreover, complete dropout from the survey was high; 57.6% of baseline respondents completed only that single survey wave (Czeisler et al., 2021b ).…”

“…Participation declined from 51.9% at the first follow‐up to 28.5% at the third follow‐up survey (Czeisler et al., 2021b ). Moreover, complete dropout from the survey was high; 57.6% of baseline respondents completed only that single survey wave (Czeisler et al., 2021b ). We concur with concerns raised by (Czeisler et al., 2021a ) that problems relating to retention of respondents over time tempers optimism and confidence in findings emerging from some longitudinal mental health surveys conducted during the pandemic.…”