Implementation of a Workplace-Based Assessment System to Measure Performance of the Core Entrustable Professional Activities in the Pediatric Clerkship

Rodgers, Victoria; Tripathi, Jaya; Lockeman, Kelly; Helou, Marieka; Lee, Clifton; Ryan, Michael S.

doi:10.1016/j.acap.2020.09.016

Cited by 11 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in a study that evaluated the operationalization of an assessment to measure the performance of core Entrustable Professional Activities (EPAs) in the pediatrics clerkship, it was apparent that students self-selected completion of EPAs that they thought would favorably affect their summative evaluation. 11 In this study, on receipt of informal feedback, there were learners who requested more sessions on learning how to obtain histories and perform physical examinations in order to master these skills. There were several learners though who requested more sessions in order to perform successfully on the Direct Observations.…”

Section: Discussionmentioning

confidence: 99%

Transition From Normative to Criterion-Based Grading in the Obstetrics and Gynecology Clerkship

Abraham

2024

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

OBJECTIVES To compare grades, National Board of Medical Examiners (NBME) Shelf Exam scores, and student satisfaction with the Obstetrics and Gynecology (OB/GYN) clerkship after transitioning from normative to criterion-based grading. METHODS Between July 2021 and July 2022, the Icahn School of Medicine at Mount Sinai (ISMMS) adhered to a normative grading scheme in which ∼60% of students achieved a grade of Honors, 30% achieved a grade of High Pass and 10% achieved a grade of Pass for the OB/GYN clerkship. In July 2022, ISMMS transitioned to a criterion-based scheme. In this scheme, 6 competencies were created. Criteria were determined for each competency, delineating achieving a score of “Pass” versus “Honors” for the specific objective. Students needed to meet the criteria for Honors for 4 out of 6 of the competencies in order to ultimately receive a grade of Honors for the clerkship. The number of students achieving Honors, NBME shelf exam scores, and student clerkship satisfaction ratings between the normative and criterion-based schemes were compared. RESULTS The number of students studying in academic year (AY) 2021–2022 and AY 2022–2023 were 134 and 137, respectively. A significantly lower percentage of students received Honors in AY 2021–2022 than in AY 2022–2023 (66% vs. 96%, P < .01). Mean exam scores were significantly higher for those receiving Honors in AY 2021–2022 than in AY 2022–2023 ( P < .05); scores for AY 2021–2022 and AY 2022–2023 were 78.9, 95% CI [77.6, 80.1] and 76.7, 95% CI [75.6, 77.8], respectively. Mean exam scores for all students were not significantly different between the 2 academic cohorts (77.8 vs. 76.2, P = .06). Clerkship satisfaction rating was significantly higher in AY 2022–2023 than in AY 2021–2022 (4.1 vs. 3.7, P < .05). CONCLUSIONS These findings support a paradigm that compares learner performance to predefined measures as opposed to peer performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Transition From Normative to Criterion-Based Grading in the Obstetrics and Gynecology Clerkship

Abraham

2024

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

show abstract

“…Based on AAMC Core EPA pilot study data from 2017–2010, only 4% of students received ≥4 workplace‐based assessments for EPA 10; however, 79% of students received an entrustment decision on this EPA 10 . Recent studies have acknowledged challenges in implementing assessments for EPA 10 compared to other EPAs, 29 with some studies reporting the exclusion of EPA 10 from workplace‐based assessment altogether 30 . To provide accurate longitudinal assessment of students, it is imperative for medical schools to design innovative ways to assess EPA 10 core functions, as the recognition and initial management of emergent conditions is a core learning outcome for medical students.…”

Section: Discussionmentioning

confidence: 99%

Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

Miller

Gibb

Caretta‐Weyer

et al. 2022

AEM Education and Training

View full text Add to dashboard Cite

Objectives:The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment.Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management.Methods: This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists.Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests.Results: Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p < 0.001, respectively. The kappa coefficients were 0.909 (n = 2698, p < 0.001) and 0.931 (n = 1872, p < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case (p < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case (p < 0.001). Conclusion:A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10

show abstract

“…11,12 In contrast, when the same scales have been applied in internal medicine, the results are more conflicting. 13,37 Hatala and colleagues found challenges using entrustment language due to the infrequency with which cognitive tasks are directly observed in practice. 39 With respect to the Core EPAs, there are some that are simply not conducive to direct observation in the workplace.…”

Section: Measurement: Performance Of Ottawa and Chen Scalesmentioning

confidence: 99%

“…The WBA system we used to generate data for this study has been described previously in detail. 34,37 In brief, all medical students were required to request a specified number of direct observations using a mobile-friendly WBA instrument linked to the learning management system. Clerkship directors identified the Core EPAs most appropriate to observe in their respective clerkship.…”

Section: Wba Systemmentioning

confidence: 99%

A Question of Scale? Generalizability of the Ottawa and Chen Scales to Render Entrustment Decisions for the Core EPAs in the Workplace

Ryan¹,

Khamishon²,

Richards³

et al. 2021

Academic Medicine

Self Cite

View full text Add to dashboard Cite

PurposeAssessments of the Core Entrustable Professional Activities (Core EPAs) are based on observations of supervisors throughout a medical student's progression toward entrustment. The purpose of this study was to compare generalizability of scores from 2 entrustment scales: the Ottawa Surgical Competency Operating Room Evaluation (Ottawa) scale and an undergraduate medical education supervisory scale proposed by Chen and colleagues (Chen). A secondary aim was to determine the impact of frequent assessors on generalizability of the data.

show abstract

Implementation of a Workplace-Based Assessment System to Measure Performance of the Core Entrustable Professional Activities in the Pediatric Clerkship

Cited by 11 publications

References 20 publications

Transition From Normative to Criterion-Based Grading in the Obstetrics and Gynecology Clerkship

Transition From Normative to Criterion-Based Grading in the Obstetrics and Gynecology Clerkship

Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

A Question of Scale? Generalizability of the Ottawa and Chen Scales to Render Entrustment Decisions for the Core EPAs in the Workplace

Contact Info

Product

Resources

About