Assessing simulation-based clinical training: comparing the concurrent validity of students' self-reported satisfaction and confidence measures against objective clinical examinations

Abstract

IntroductionSimulation-based education (SBE) literature is replete with student satisfaction and confidence measures to infer educational outcomes. This research aims to test how well students' satisfaction and confidence measures correlate with expert assessments of students' improvements in competence following SBE activities.MethodsN=85 paramedic students (mean age 23.7 years, SD=6.5; 48.2% female) undertook a 3-day SBE workshop. Students' baseline competence was assessed via practical scenario simulation assessments (PSSAs) administered by expert paramedics and confidence via a questionnaire. Postworkshop competence and confidence plus self-reported students' satisfaction were remeasured.ResultsPSSA scores increased significantly between baseline and post workshop (35.7%→53.4%, p<0.001), as did students' confidence (55.7%→60.5%, p<0.001), and their workshop satisfaction was high (71.0%). Satisfaction and postworkshop confidence measures were moderately correlated (r=0.377, p=0.001). However, competence improvements were not significantly correlated with either satisfaction (r=−0.107 p=0.344) or change in confidence (r=−0.187 p=0.102).DiscussionStudents' self-reported satisfaction and confidence measures bore little relation to expert paramedics' judgements of their educational improvements. Satisfaction and confidence measures appear to be dubious indicators of SBE learning outcomes.

The Effect of Simulation Training on Nurses and Intern Nursing Students’ Skill, Confident and Satisfaction Regarding Neonatal Resuscitation

“…On the other hand this results are congruent with (Jefferies, 2007) which suggests that the outcomes of satisfaction and self-confidence are due to combination of factors related to demographic characteristics. Also the study carry by (Carter, et al, 2015) reported that there was statistical significantly between baseline and post workshop (35.7%→53.4%, p<0.001), as did students' confidence (55.7%→60.5%, p<0.001), and their workshop satisfaction was high (71.0%). Satisfaction and post workshop confidence measures were moderately correlated (r=0.377, p=0.001).…”

Analysis of hot spots and frontiers of nursing scenario simulation teaching research at home and abroad

“…Abe, Y et al (e.g., Abe, Y et al, 2013 ) used the teamwork activity inventory to assess students’ non-technical skills. Meanwhile, some scholars pointed out that satisfaction and confidence are valid indicators for measuring the teaching of scenario-based simulations (e.g., Carter, O et al, 2016 ). While domestic scholars Guo S. Y et al (e.g., Guo S. Y et al, 2022 ) used a fuzzy comprehensive evaluation method to assess the teaching quality of nursing scenario simulation practical training.…”

Recorded performance during simulation activities in medical imaging: do students see a benefit?

“…Overall student perceptions of benefit were particularly strong in the areas of communication, efficiency and teamwork and meet the outcomes identified by Carter et al (2016 p. 30) that ‘student satisfaction and improvements in confidence are highly desirable educational outcomes from a pedagogical perspective.’3 …”