The Reliability and Validity of the American Board of Internal Medicine Monthly Evaluation Form

Durning, Steven J.; Cation, Lannie J.; Jackson, Jeffrey L.

doi:10.1097/00001888-200311000-00021

Cited by 19 publications

(19 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We had previously shown that both the immediately two preceding versions of the medicine resident evaluation forms had poor validity and reliability. 35 Whether assessments based on the new ACGME Internal Medicine Milestones 36 will truly improve the evaluation process remains to be seen.…”

Section: Discussionmentioning

confidence: 99%

The Quality of Written Feedback by Attendings of Internal Medicine Residents

et al. 2015

View full text Add to dashboard Cite

BACKGROUND: Attending evaluations are commonly used to evaluate residents. OBJECTIVES: Evaluate the quality of written feedback of internal medicine residents. DESIGN: Retrospective. PARTICIPANTS: Internal medicine residents and faculty at the Medical College of Wisconsin from 2004 to 2012. MAIN MEASURES: From monthly evaluations of residents by attendings, a randomly selected sample of 500 written comments by attendings were qualitatively coded and rated as high-, moderate-, or low-quality feedback by two independent coders with good inter-rater reliability (kappa: 0.94). Small group exercises with residents and attendings also coded the utterances as high, moderate, or low quality and developed criteria for this categorization. In-service examination scores were correlated with written feedback. KEY RESULTS: There were 228 internal medicine residents who had 6,603 evaluations by 334 attendings. Among 500 randomly selected written comments, there were 2,056 unique utterances: 29 % were coded as nonspecific statements, 20 % were comments about resident personality, 16 % about patient care, 14 % interpersonal communication, 7 % medical knowledge, 6 % professionalism, and 4 % each on practice-based learning and systems-based practice. Based on criteria developed by group exercises, the majority of written comments were rated as moderate quality (65 %); 22 % were rated as high quality and 13 % as low quality. Attendings who provided high-quality feedback rated residents significantly lower in all six of the Accreditation Council for Graduate Medical Education (ACGME) competencies (p <0.0005 for all), and had a greater range of scores. Negative comments on medical knowledge were associated with lower in-service examination scores. CONCLUSIONS: Most attending written evaluation was of moderate or low quality. Attendings who provided highquality feedback appeared to be more discriminating, providing significantly lower ratings of residents in all six ACGME core competencies, and across a greater range. Attendings' negative written comments on medical knowledge correlated with lower in-service training scores.

show abstract

Section: Discussionmentioning

confidence: 99%

The Quality of Written Feedback by Attendings of Internal Medicine Residents

et al. 2015

View full text Add to dashboard Cite

show abstract

“…In a previous study, we examined the relationship between ITE scores and ABIM scores for internal medicine residents 15 and found a weak correlation between the two, although residents in the bottom quartile of the ITE were at increased risk of failing the ABIM certification. That study had several limitations, most notably a small study population of 71 residents and just 4 residents who failed the ABIM-CE.…”

Section: Methodsmentioning

confidence: 99%

The Relationship Between Internal Medicine Residency Graduate Performance on the ABIM Certifying Examination, Yearly In-Service Training Examinations, and the USMLE Step 1 Examination

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…These studies, in turn, built upon previous research showing that clinical performance assessments of learners and clinical teaching assessments, which were originally believed to tap multiple dimensions, measured the broader attributes of humanism and clinical skills (McLeod et al 1993;Durning et al 2003;Silber et al 2004). …”

Section: Conceptual Framework and Qualitative Methodsmentioning

confidence: 99%

Resident physicians’ perspectives on effective outpatient teaching: a qualitative study

Kisiel

Bundrick

Beckman

2009

Adv in Health Sci Educ

View full text Add to dashboard Cite

Learning theories, which suggest that experienced faculty use collaborative teaching styles, are reflected in qualitative studies of learners in hospital settings. However, little research has used resident focus groups to explore characteristics of successful teachers in outpatient clinics. Therefore, focus group discussions with first through third-year internal medicine residents at a large academic medical center were conducted to better understand residents' perspectives on effective outpatient teaching. A group facilitator solicited residents' reflections, based on their lived experiences, on teaching domains from previous factor analytic studies: interpersonal, clinical-teaching, and efficiency. Researchers coded focus group transcripts and identified themes within the domains. Final themes were determined by consensus. Leading themes were "kindness" and "teacher-learner relationships." Junior residents were sensitive to faculty who were brusque, harsh, and degrading. Senior residents respected faculty who were humble, collaborative, and allowed residents to co-manage teaching encounters. Seniors emphasized the importance of faculty role-modelling and preferentially staffed with experts to "gain wisdom from experience." Overall, residents expressed that effective learning requires grounded teacher-learner relationships. These findings support learning theories and previous factor analytic studies. However, this qualitative study provided insights that could not be gleaned from assessment scores alone.

show abstract

The Reliability and Validity of the American Board of Internal Medicine Monthly Evaluation Form

Abstract: The ABIM-MEF appears to be reliable and valid. Further, factor analysis results support the ABIM's movement to simplify the monthly evaluation form to the new Accreditation Council for Graduate Medical Education core competencies.

Cited by 19 publications

References 8 publications

The Quality of Written Feedback by Attendings of Internal Medicine Residents

The Quality of Written Feedback by Attendings of Internal Medicine Residents

The Relationship Between Internal Medicine Residency Graduate Performance on the ABIM Certifying Examination, Yearly In-Service Training Examinations, and the USMLE Step 1 Examination

Resident physicians’ perspectives on effective outpatient teaching: a qualitative study

Contact Info

Product

Resources

About