It’s Getting Late: Improving Completion Rates in a Hard-to-Reach Sample

Abstract

Maintaining high retention rates in longitudinal survey research is critical to ensure study success and prevent bias. However, retaining study participants, particularly youth and young adults, presents challenges. This study aims to assess the efficacy of various monetary and communication strategies in retaining members of the Truth Longitudinal Cohort, a probability-based, national online panel of youth and young adults. Baseline assessments were conducted in 2014, and follow-up assessment occurred six months later. A total of 4,467 respondents did not complete the follow-up survey after approximately nine weeks and were then randomized into one of 18 experimental conditions with varying combinations of monetary incentives ($30, $40, and $50) and communications (interactive voice recording, live phone prompt, email, and postcard). Overall, 15.9% of participants assigned to the experiment completed the survey. Results indicated no significant difference in response across the three incentive conditions after adjusting for multiple comparisons. Results also showed that using multiple methods of communication was more efficacious than any single method. However, when using single methods of communication to prompt response, a mailed postcard prompt was more efficacious than any other method tested after adjusting for multiple comparisons. The findings from this study can inform the design of future strategies to reduce panel attrition and improve retention rates. introduction High retention rates in longitudinal survey research are key to ensuring study success. Sample retention helps ensure robust power and variability to detect study effects. These issues are of particular concern when assessing outcome behaviors with relatively low incidence, such as tobacco use among youth. Strong retention also minimizes bias due to differential attrition. Such bias is common in studies that examine risk behaviors, as those with higher levels of risk are more likely to be lost to follow up (Morrison et al. 1997). Retaining study participants over time can be especially difficult with teen and young adult populations (Price et al. 2016). Obtaining parental consent for respondents younger than 18 and increasing mobility and life course changes

Recruiting and Retaining Youth and Young Adults in the Policy and Communication Evaluation (PACE) Vermont Study: Randomized Controlled Trial of Participant Compensation (Preprint)

“…The higher proportion of female participants enrolled in the PACE Vermont Study is consistent with other studies, documenting higher recruitment of women to health studies via web-based advertising [38,40,41]. Retention in this web-based cohort study was over 70% at a 6-month follow-up and did not differ by the incentive condition, as seen in other studies of young adults [27]. Participant feedback on the study experience was positive.…”

“…Retention in our cohort of youth and young adults was higher at 6 months (72.18%) compared with a national cohort of young adults aged 15 to 21 years who also completed web-based surveys (63%) [27]. Our randomized experiment regarding incentive conditions showed, similar to previous studies [37][38][39][40], that multiple means of compensation produce equal retention when combined with providing completion bonuses and sending multiple reminders.…”

“…Adoption of web-based surveys for data collection is likely to appeal to young people, who grew up with computers and use them in virtually all aspects of their lives, while reducing transportation and other costs that serve as barriers to engaging in traditional clinical trials [21][22][23][24]. With respect to retaining young people in longitudinal studies, there was no clear recommendation on incentives to maximize retention, as previous studies had identified multiple means, including increasing participant payments, conducting sweepstakes, providing bonuses, and sending reminder postcards [25][26][27][28].…”

Recruiting and Retaining Youth and Young Adults in the Policy and Communication Evaluation (PACE) Vermont Study: Randomized Controlled Trial of Participant Compensation

“…The higher proportion of female participants enrolled in the PACE Vermont Study is consistent with other studies, documenting higher recruitment of women to health studies via web-based advertising [ 38 , 40 , 41 ]. Retention in this web-based cohort study was over 70% at a 6-month follow-up and did not differ by the incentive condition, as seen in other studies of young adults [ 27 ]. Participant feedback on the study experience was positive.…”

“…Retention in our cohort of youth and young adults was higher at 6 months (72.18%) compared with a national cohort of young adults aged 15 to 21 years who also completed web-based surveys (63%) [ 27 ]. Our randomized experiment regarding incentive conditions showed, similar to previous studies [ 37 - 40 ], that multiple means of compensation produce equal retention when combined with providing completion bonuses and sending multiple reminders.…”