Optimizing Survey Response Rates in Graduate Medical Education Research Studies

Abstract

Background and Objective: Survey response rates of 70% or higher are needed if findings are to be considered generalizable. Unfortunately, survey studies of health professionals have declining response rates. We have conducted survey research with residents and residency directors for over 13 years. Here we describe the strategies we used to obtain optimal response rates in residency training research collaboratives. Methods: We administered over 6,000 surveys between 2007 and 2019 to evaluate the Preparing the Personal Physician for Practice and Length of Training Pilot studies, both of which involved redesigning residency training. Survey recipients included program directors, clinic managers, residents, graduates, as well as supervising physicians and clinic staff members. We logged and analyzed survey administration efforts and approaches to optimize strategies. Results: Overall, we obtained response rates of 100% for program director surveys, 98% for resident surveys, 97% for continuity clinic surveys, 81% for graduates surveys, and 48% for the supervising physician and 43% for the clinic staff. Response rates were highest when the relationships between the evaluation team and survey recipients were closest. Strategies for optimizing response rates included (1) building relationships with all participants whenever possible, (2) sensitivity to survey timing and fatigue, and (3) using creative and persistent follow-up measures to encourage survey completion. Conclusion: High response rates are achievable, though they require an investment in time, resources, and ingenuity in connecting with study populations. Investigators conducting survey research must consider administrative efforts needed to achieve target response rates, including planning for funds accordingly.

Assessing the Knowledge and Comfort Level of US Family Medicine Residents Regarding Consenting and Confidentiality in Adolescent Health Care

“…Higher response rates may have been achieved by using monetary incentives, multiple recruitment methods, and additional data collection methods, such as paper mailings. 26 Furthermore, the volunteer sampling method used for participant selection may have introduced selection bias, and thus the generalizability of our findings is limited. In addition, although results showed that residents who receive formal or informal training were more comfortable providing adolescent health care and were more likely to conduct time alone with adolescents than those who did not receive training, the specific type of training received was not explored.…”

“…This low response rate likely reflects the single recruitment method and electronic strategy. Higher response rates may have been achieved by using monetary incentives, multiple recruitment methods, and additional data collection methods, such as paper mailings 26 . Furthermore, the volunteer sampling method used for participant selection may have introduced selection bias, and thus the generalizability of our findings is limited.…”

Knowledge and Behavior of Primary Care Physicians Regarding Utilization of Standardized Tools in Screening and Assessment of Anxiety, Depression, and Mood Disorders at a Large Integrated Health System

“…The response rate and demographics of the participants of this study, as shown in the results section above, are comparable to similar large-scale national surveys. 21 …”

Captive Samples Are Not the Answer to Survey Response Rates

“…I was eager to read Ericson et al's "Optimizing Survey Response Rates in Graduate Medical Education Research Studies," which described potential strategies to optimize response rates to survey research. 1 Unfortunately, the authors' described strategies are not generalizable because of their highly motivated sample populations. The residency programs involved in the Preparing the Personal Physician for Practice and Length of Training Pilot studies went through rigorous application processes to participate and fully understood the expectations for studying their experience.…”

“…Survey response rates in medical education research have steadily declined. 1 Despite this decline, though, the three highest-impact journals for medical education (Academic Medicine, Medical Education, and Advances in Health Sciences Education) do not require a minimum survey response rate in studies submitted for publication. 2 This lack of a requirement may stem from prior research suggesting that response rate does not inversely correlate with nonresponse bias and that justification of the target population sample better determines the relevance of the data.…”