What is the value of simulated patient assessment in structured clinical examinations of undergraduate students, and how should it be used?

Abstract

Introduction:The aim of this study was to explore correlations between clinical assessor and simulated patient (SP) scores drawn from summative Integrated Structured Clinical Examination (ISCE) and inform the best use of SP scores in future assessments. Materials and Methods:This retrospective study explores summative clinical assessor and formative SP numeric scores drawn from summative ISCE assessments spanning three academic years (2017-18 to 2019-20). Analyses were carried out using R 3.5.1 (R Core Team, 2018), with the stats package. Results:The sample consisted of 169 final-year BDS students across the three cohorts and included 95 females (56.2%) and 74 males (43.8%). Data from eight substations where SPs were included, were explored. Kendall's Tau, a non-parametric correlation, was used to investigate the relationships between the assessor and SP scores. Clinical assessor scores were out of a total of 20 points across various assessed domains within each substation. The formative SP assessment was out of 10 points with the same five affective domains related to communication included in each substation. Overall, the assessor and patient substation scores were not correlated (τ = 0.04, p = .272) indicating that communication skills alone, as assessed by patients, do not correlate with more holistic performance across other domains.There was significant positive correlation for two of the eight substations with the other substations showing very little correlation. Conclusion:This study shows that assessment of student performance by SPs does not show a correlation with examiner scores and may provide additional information relating to affective skills of students. Notwithstanding the limitations of this study, the findings underscore the need to investigate further the value of involvement of SPs in clinical assessments to explore if scores by SPs can be used to enhance the validity of assessments if used summatively.

Assessment of communication and physical exam skills: A comparison of students, faculty and standardized patients

“…This may reflect that instructors tend to be focused on the objective, "how to do" tasks. 19 It also suggests that perhaps at this stage of the training, and due to the limited patient care involvement, students require more exposure with instruction and repetition as to what the ideal expected outcomes are.…”

“…34,35 Thus, as also described by Ali et al, this underscores the importance of including SPs for assessment of students' soft skills as they may be able to recognize skills that are key in the development of a patient-provider relationship and that can contribute to fostering person-centered care. 19 As for the differences in each specific criterion between students and SPs assessment, these were seen in three elements pertaining to communication skills: the student used appropriate eye contact; the student exhibits an acceptable tone of voice and rate of speech and the student displays a professional appearance. In those instances, students graded themselves in a more negative way com-pared to SPs.…”

“…On the other hand, the students graded themselves less positively compared to faculty in the following criteria: the student exhibits an acceptable tone of voice and rate of speech and, the student displays a professional appearance. This may reflect that instructors tend to be focused on the objective, “how to do” tasks 19 . It also suggests that perhaps at this stage of the training, and due to the limited patient care involvement, students require more exposure with instruction and repetition as to what the ideal expected outcomes are.…”

“…16,17 Moreover, studies that illustrate the differences between faculty and SPs grading of students are limited, with some studies reporting alignment 18 and others, disagreement. 19 Most of these reports pertain to the use of SPs in medical, pharmacy, and nursing education, with limited data in dental education. Consequently, studies that report on outcomes of dental schools' instructional activities involving the use of Faculty, SPs, and students' self-evaluation practices are needed to provide dental educators with appropriate means to make decisions as to the benefit, or lack thereof, of including assessments from these different perspectives in the curriculum.…”

“…Nevertheless, studies have shown that the evaluation of health care professional students by instructors and SPs can differ from those completed by students following a self‐assessment practice 16,17 . Moreover, studies that illustrate the differences between faculty and SPs grading of students are limited, with some studies reporting alignment 18 and others, disagreement 19 . Most of these reports pertain to the use of SPs in medical, pharmacy, and nursing education, with limited data in dental education.…”