Improving the assessment of communication competencies in a national licensing OSCE: lessons learned from an experts’ symposium

Abstract

Background: As the communication competencies of physicians are crucial for providing optimal patient care, their assessment in the context of the high-stakes Objective Structured Clinical Examination (OSCE) is of paramount importance. Despite abundant literature on the topic, evidence-based recommendations for the assessment of communication competencies in high stakes OSCEs are scarce. As part of a national project to improve communication-competencies assessments in the Swiss licensing exam, we held a symposium with national and international experts to derive corresponding guidelines. Methods: Experts were invited on account of their recognized expertise either in teaching or assessing communication competencies, or in conducting national high-stakes OSCEs. They were asked to propose concrete solutions related to four potential areas for improvement: the station design, the rating tool, the raters' training, and the role of standardized patients. Data gene.rated in the symposium was available for analysis and consisted of video recordings of plenary sessions, of the written summaries of group work, and the cards with participants' personal take-home messages. Data were analyzed using a thematic analysis approach. Results: Nine major suggestions for improving communication-competencies assessments emerged from the analysis and were classified into four categories, namely, the roles of the OSCE scenarios, rating tool, raters' training, and simulated patients. Conclusion: In the absence of established evidence-based guidelines, an experts' symposium facilitated the identification of nine practical suggestions for improving the assessment of communication competencies in the context of high-stakes OSCEs. Further research is needed to test effectiveness of the suggestions and how they contribute to improvements in the quality of high-stakes communication-competencies assessment.

Assessing communication skills during OSCE: need for integrated psychometric approaches

“…In Switzerland, this scale is currently used in the context of the Federal Licensing Exam, where it showed good internal consistency among the four dimensions [25], as in other previous research within OSCE's settings [20][21][22][23]. The selection of this instrument at the Swiss national level derives from few essential considerations: 1) the state-of-the-art of instructional methods and assessments for CS across Swiss medical schools; 2) the direct use of the instrument without the need for specific training; 3) the broad coverage of communication competencies being therefore applicable to different medical faculties independently by their curriculum; and 4) the ability to complete the assessment of CS in less than 2 min [25]. Nevertheless, to the best of the authors' knowledge, no study has yet evaluated the invariance of the factorial structure of this scale across stations, or applied IRT to it.…”

Perspective matters: assessment of medical students' communication and interpersonal skills by simulated patients from the internal and external patient perspective

“…Rating of medical students’ interpersonal skills or empathy in communication situations with simulated patients can be carried out in different ways: by only examiners directly after the consultation [ 14 ], [ 15 ], by the simulated patients themselves and by examiners directly after the consultation [ 6 ], [ 7 ], [ 8 ], [ 9 ], or by examiners watching video recorded communication situations at a convenient time [ 16 ], [ 17 ]. In our study, simulated patients rated the same consultations from the patient perspective directly after the consultation and four months later from the external perspective of the patient’s view, watching their videos.…”

Long experience with a web-based, interactive, conversational virtual patient case simulation for medical students’ evaluation: comparison with oral examination

“…However, OSCEs are resource intensive [1], and cannot be used routinely by many faculties of medicine. Also, when OSCE evaluators are not well trained and/ or are not used on a frequent and continuous basis, their reliability and validity may be jeopardized [2,3].…”