The Promise of Milestones: Are They Living Up to Our Expectations?

Li, Su-Ting Terry

doi:10.4300/jgme-d-16-00694.1

Cited by 25 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Discussionmentioning

confidence: 99%

“…The ability to correlate target milestone levels to success in independent clinical practice would allow for establishment of more definitive graduation benchmarks for PDs. 18 A group of family practice physicians developed milestones for faculty based on the ACGME core competencies. 19 In developing and validating these milestone criteria, they used a combination of levels 3 and 4 from the ACGME SCs as goal level for faculty at the assistant professor level.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Pediatric emergency medicine fellows’ milestone evaluations: Do they all meet the targets for graduation?

Roskind

Leonard

Baghdassarian

et al. 2021

AEM Education and Training

View full text Add to dashboard Cite

Background:The ACGME Milestone Project created a competency-based trainee assessment tool. Subcompetencies (SCs) are scored on a 5-point scale; level 4 is recommended for graduation. The 2018 Milestones Report found that across subspecialties, not all graduates attain level 4 for every SC. Objective: The objective was to describe the number of pediatric emergency medicine (PEM) fellows who achieve ≥ level 4 in all 23 SCs at graduation and identify SCs where level 4 is not achieved and factors predictive of not achieving a level 4. Methods: This is a multicenter, retrospective cohort study of PEM fellows from 2014 to 2018. Program directors provided milestone reports. Descriptive analysis of SC scores was performed. Subanalyses assessed differences in residency graduation scores, first-year fellowship scores, and the rate of milestone attainment between fellows who did and did not attain ≥ level 4 at graduation.Results: Data from 392 fellows were obtained. There were no SCs in which all fellows attained ≥ level 4 at graduation; the range of fellows scoring < level 4 per SC was 7% to 39%. A total of 67% of fellows did not attain ≥ level 4 on one or more SC. While some fellows failed to attain ≥ level 4 on up to all 23 SCs, 26% failed to meet level 4 on only one or two. In 19 SCs, residency graduation and/or first year fellow scores were lower for fellows who did not attain ≥ level 4 at graduation compared to those who did (mean difference = 0.74 points). Among 10 SCs, fellows who did not attain ≥ level 4 at graduation had a faster rate of improvement compared to those who did attain ≥ level 4. Conclusion:In our sample, 67% of PEM fellows did not attain level 4 for one or more of the SCs at graduation. Low scores during residency or early in fellowship may predict difficulty in meeting level 4 by fellowship completion.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Pediatric emergency medicine fellows’ milestone evaluations: Do they all meet the targets for graduation?

Roskind

Leonard

Baghdassarian

et al. 2021

AEM Education and Training

View full text Add to dashboard Cite

show abstract

“…We assessed convergent validity using the residents’ ACGME Milestones scores; however, ACGME Milestones scores have been suggested to lack inter-rater reliability, and consistency between GME programs. 38 – 39 Moreover, the ACGME Milestones are not a complete determination of residents’ abilities nor do they assess all areas essential to unsupervised practice. 40 Finally, the ACGME Milestones were not designed to be an assessment tool in this context.…”

Section: Limitationsmentioning

confidence: 99%

Assessment of Emergency Medicine Residents’ Clinical Reasoning: Validation of a Script Concordance Test

Steinberg

Cowan

Lin

et al. 2020

WestJEM

View full text Add to dashboard Cite

Introduction A primary aim of residency training is to develop competence in clinical reasoning. However, there are few instruments that can accurately, reliably, and efficiently assess residents’ clinical decision-making ability. This study aimed to externally validate the script concordance test in emergency medicine (SCT-EM), an assessment tool designed for this purpose. Methods Using established methodology for the SCT-EM, we compared EM residents’ performance on the SCT-EM to an expert panel of emergency physicians at three urban academic centers. We performed adjusted pairwise t-tests to compare differences between all residents and attending physicians, as well as among resident postgraduate year (PGY) levels. We tested correlation between SCT-EM and Accreditation Council for Graduate Medical Education Milestone scores using Pearson’s correlation coefficients. Inter-item covariances for SCT items were calculated using Cronbach’s alpha statistic. Results The SCT-EM was administered to 68 residents and 13 attendings. There was a significant difference in mean scores among all groups (mean + standard deviation: PGY-1 59 + 7; PGY-2 62 + 6; PGY-3 60 + 8; PGY-4 61 + 8; 73 + 8 for attendings, p < 0.01). Post hoc pairwise comparisons demonstrated that significant difference in mean scores only occurred between each PGY level and the attendings (p < 0.01 for PGY-1 to PGY-4 vs attending group). Performance on the SCT-EM and EM Milestones was not significantly correlated (r = 0.12, p = 0.35). Internal reliability of the exam was determined using Cronbach’s alpha, which was 0.67 for all examinees, and 0.89 in the expert-only group. Conclusion The SCT-EM has limited utility in reliably assessing clinical reasoning among EM residents. Although the SCT-EM was able to differentiate clinical reasoning ability between residents and expert faculty, it did not between PGY levels, or correlate with Milestones scores. Furthermore, several limitations threaten the validity of the SCT-EM, suggesting further study is needed in more diverse settings.

show abstract

“…In 2013, the Accreditation Council for Graduate Medical Education (ACGME) released the Emergency Medicine (EM) milestones to help delineate the progress of a resident in attaining skills in each competency domain and enhancing their assessment and feedback for improvement. 1 These milestones have become the standard for guiding resident assessment and have been used to identify individual resident improvement areas. One area that has not been standardized is a resident individual self-assessment (ISA) process.…”

Section: Introductionmentioning

confidence: 99%

Resident Self-Assessment and the Deficiency of Individualized Learning Plans in Our Residencies

Della-Giustina

Kamran

Wood

et al. 2021

WJEM

View full text Add to dashboard Cite

The Promise of Milestones: Are They Living Up to Our Expectations?

Cited by 25 publications

References 27 publications

Pediatric emergency medicine fellows’ milestone evaluations: Do they all meet the targets for graduation?

Pediatric emergency medicine fellows’ milestone evaluations: Do they all meet the targets for graduation?

Assessment of Emergency Medicine Residents’ Clinical Reasoning: Validation of a Script Concordance Test

Resident Self-Assessment and the Deficiency of Individualized Learning Plans in Our Residencies

Contact Info

Product

Resources

About