Patient involvement in assessment: How useful is it?

Malau‐Aduli, Bunmi S.

doi:10.1111/medu.14802

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mystery shopping has been used to investigate prescribing and pharmaceutical sales provisions (Björnsdottir et al, 2020 ; Cheo et al, 2020 ; Collins et al, 2020 ); but not student competencies. Generally, real life patients are used in WBAs (Khalife et al, 2022 ; Malau-Aduli, 2022 ) but we needed to compare performance with the same cases so we trained simulated patients to be used. Due to time constraints and mystery shopper and workplace supervisor availability, it was not possible to visit all the students within their placement period but 92/289 students is a reasonable representation of the cohort.…”

Section: Discussionmentioning

confidence: 99%

“…WBAs and OSCEs are both widely used for the assessment of clinical competence in health education, however, there has not been any literature comparing the two contexts and how students perform in a simulated versus un-simulated environment. Real life patients are commonly used in WBAs to assess competencies in the workplace (Khalife et al, 2022 ), and have found to be useful to have an authentic voice and provide more valuable feedback that prepare students for practice (Malau-Aduli, 2022 ). Our WBA used simulated patients as mystery shoppers as our study aims to compare the same clinical case in OSCEs and WBAs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Linking assessment to real life practice – comparing work based assessments and objective structured clinical examinations using mystery shopping

Lim,

Krishnan,

Singh

et al. 2023

Adv in Health Sci Educ

View full text Add to dashboard Cite

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions’ education. Underpinned by the extrapolation inference in Kane’s Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students’ OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students’ performance. A sequential explanatory mixed methods approach where a grade comparison between students’ performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a ‘mystery shopper’ and the process to ‘mystery shopping’ in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students’ provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Linking assessment to real life practice – comparing work based assessments and objective structured clinical examinations using mystery shopping

Lim,

Krishnan,

Singh

et al. 2023

Adv in Health Sci Educ

View full text Add to dashboard Cite

show abstract

Patient involvement in assessment: How useful is it?

Malau‐Aduli

2022

Medical Education

Self Cite

View full text Add to dashboard Cite

show abstract

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping

Lim,

Krishnan,

Singh

et al. 2023

Preprint

View full text Add to dashboard Cite

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions’ education. Underpinned by the extrapolation inference in Kane Validity’s Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students’ OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students’ performance. A sequential explanatory mixed methods approach where a grade comparison between students’ performance in their OSCE and WBA (conducted as a mystery shopper visit at the workplace) was performed, students were then followed up with semi-structured interviews to discuss the students’ experiences in both settings. Students were third year pharmacy undergraduates in 2022. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 39.2% lower compared to the OSCE score (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage a real-life patient. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as oppose to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

show abstract

Patient involvement in assessment: How useful is it?

Abstract: The author unveils a strategy for enquiry that can facilitate identification of best practices for involving real patients in OSCE and WBA competency‐based assessments

Cited by 3 publications

References 18 publications

Linking assessment to real life practice – comparing work based assessments and objective structured clinical examinations using mystery shopping

Linking assessment to real life practice – comparing work based assessments and objective structured clinical examinations using mystery shopping

Patient involvement in assessment: How useful is it?

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping

Contact Info

Product

Resources

About