Educator's blueprint: A how‐to guide for survey design

Abstract

Surveys are ubiquitous in medical education. They can be valuable for assessment across a wide range of applications and are frequently used in medical education research. This Educator's Blueprint paper reviews the best practices in survey design with a focus on survey development. Key components of the survey design process include determining whether a survey is the right tool, using an intentional approach to content development, and following best practices in item writing and formatting. These processes are meant to help educators and researchers design better surveys for making better decisions.

Educator's blueprint: A how‐to guide for collecting validity evidence in survey‐ based research

“…• Associations between survey scores and external variables with theoretical associations • Data from pilot testing "As expected, confidence levels generally increased by PGY; however, this was not always the case among PGY-4s whose confidence ratings were lower than the mean across several skill areas." 1…”

“…3 We have previously outlined several processes in the early stages of survey development that help build a case for content validity. 1 First, a detailed literature search can help identify previously derived survey instruments assessing the same or similar construct(s) and will help the researcher better refine the phenomena they intend to measure. 9 However, just because a survey has been published does not mean validity evidence has been collected in a rigorous manner.…”

“…Therefore, it is essential that clinician educators incorporate thoughtful design to ensure meaningful results. The first paper in this series on survey development and implementation focused on the initial steps of survey development: deciding whether a survey is the appropriate tool for assessing variables of interest, settling on the objectives of the survey, picking constructs to assess, and writing and formatting survey items 1 …”

The next generation of researchers:One‐yearoutcome data from theSAEM Advanced Research Methodology Evaluation and Design in Medical Education(ARMED MedEd) program

“…We developed a baseline and postcourse survey using best practice guidelines and targeting multiple Kirkpatrick levels. 16,17 Our research team reviewed the survey items in detail for content validity, including assessments of comprehension, retrieval, judgment, and response process. To optimize response process validity, we piloted the survey among CEs who were not participants in the study and conducted cognitive interviewing and usability testing to identify areas for improvement.…”

“…The McMaster University Research Ethics Board approved an online mixed‐methods survey of participants. We developed a baseline and postcourse survey using best practice guidelines and targeting multiple Kirkpatrick levels 16,17 . Our research team reviewed the survey items in detail for content validity, including assessments of comprehension, retrieval, judgment, and response process.…”