Internal medicine residents do not accurately assess their medical knowledge

Jones, Roger; Panda, Mukta; Desbiens, Norman A.

doi:10.1007/s10459-007-9058-2

Cited by 17 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there is an increasing push towards self-directed learning and autonomy in selection of electives it is well recognized that residents themselves are not always good judges of their own medical knowledge [15]. As in prior studies [16], our data showed strong correlation between the program director medical knowledge score and the ABIM-CE score, reinforcing the continued predictive value of program director evaluations, and supporting use of these evaluations in guiding residents throughout residency.…”

Section: Discussionsupporting

confidence: 82%

Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination

et al. 2012

View full text Add to dashboard Cite

BackgroundThe American Board of Internal Medicine Certification Examination (ABIM-CE) is one of several methods used to assess medical knowledge, an Accreditation Council for Graduate Medical Education (ACGME) core competency for graduating internal medicine residents. With recent changes in graduate medical education program directors and internal medicine residents are seeking evidence to guide decisions regarding residency elective choices. Prior studies have shown that formalized elective curricula improve subspecialty ABIM-CE scores. The primary aim of this study was to evaluate whether the number of subspecialty elective exposures or the specific subspecialties which residents complete electives in impact ABIM-CE scores.MethodsABIM-CE scores, elective exposures and demographic characteristics were collected for MedStar Georgetown University Hospital internal medicine residents who were first-time takers of the ABIM-CE in 2006–2010 (n=152). Elective exposures were defined as a two-week period assigned to the respective subspecialty. ABIM-CE score was analyzed using the difference between the ABIM-CE score and the standardized passing score (delta-SPS). Subspecialty scores were analyzed using percentage of correct responses. Data was analyzed using GraphPad Prism version 5.00 for Windows.ResultsPaired elective exposure and ABIM-CE scores were available in 131 residents. There was no linear correlation between ABIM-CE mean delta-SPS and the total number of electives or the number of unique elective exposures. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures (143.4 compared to 129.7, p=0.051). Repeated electives in individual subspecialties were not associated with significant difference in mean ABIM-CE delta-SPS.ConclusionsThis study did not demonstrate significant positive associations between individual subspecialty elective exposures and ABIM-CE mean delta-SPS score. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures suggesting there may be an “ideal” number of elective exposures that supports improved ABIM-CE performance. Repeated elective exposures in an individual specialty did not correlate with overall or subspecialty ABIM-CE performance.

show abstract

Section: Discussionsupporting

confidence: 82%

Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Numerous studies have suggested that physicians are poor self-assessors based on a limited correlation between physician's self-assessment and expert observer ratings (Barnsley et al 2004;Biernat et al 2003;Claridge et al 2003;Jones et al 2008;Leopold et al 2005;Parker et al 2004). These findings have led some authors to conclude that physicians should rely less on self-assessment and more on external assessment (Davis et al 2006).…”

Section: Introductionmentioning

confidence: 99%

Understanding self-assessment as an informed process: residents’ use of external information for self-assessment of performance in simulated resuscitations

Plant

Corden

Mourad

et al. 2012

Adv in Health Sci Educ

View full text Add to dashboard Cite

Self-directed learning requires self-assessment of learning needs and performance, a complex process that requires collecting and interpreting data from various sources. Learners' approaches to self-assessment likely vary depending on the learner and the context. The aim of this study was to gain insight into how learners process external information and apply their interpretation of this information to their self-assessment and learning during a structured educational activity. The study combined quantitative performance data with qualitative interview data. Pediatric residents led video-recorded simulated resuscitations and rated their crisis resource management skills on a validated 6-item instrument. Three independent observers rated the videos using the same instrument. During semi-structured interviews, each resident reviewed the video, rerated performance, discussed the self-assessment process, and interpreted feedback and observer scores. Transcripts were analyzed for themes. Sixteen residents participated. Residents' self-assessed scores ranged widely but usually fell within two points of the observers. They almost universally lowered their scores when self-assessing after the video review. Five major themes emerged from qualitative analysis of their interviews: (1) residents found self-assessment important and useful in certain contexts and conditions; (2) residents varied in their self-directed learning behaviors after the simulated resuscitation; (3) quantitative observer assessment had limited usefulness; (4) video review was difficult but useful; and (5) residents focused on their weaknesses and felt a need for constructive feedback to enhance learning. The residents in our study almost uniformly embraced the importance of self-assessment for all medical professionals. Even though video review had a negative impact on their self-assessment scores and was perceived as painful, residents saw this as the most useful aspect of the study exercises residents. They were less accepting of the quantitative assessment by observers. Residents explained their tendency to focus on weaknesses as a way to create an incentive for learning, demonstrating that self-assessment is closely linked to self-directed learning. How learners can use video review and external assessment most effectively to guide their self-directed learning deserves further study.

show abstract

“…There is the suggestion that this may be true. Studies done in orthopedic residency programs have shown that performance on USMLE Step 1 and 2 correlated with that on the Orthopedic In-training examination (Black et al 2006;Jones et al 2008).…”

mentioning

confidence: 99%

Correlation of United States Medical Licensing Examination and Internal Medicine In-Training Examination performance

Perez

Greer

2009

Adv in Health Sci Educ

View full text Add to dashboard Cite

The Internal Medicine In-Training Examination (ITE) is administered during residency training in the United States as a self-assessment and program assessment tool. Performance on this exam correlates with outcome on the American Board of Internal Medicine Certifying examination. Internal Medicine Program Directors use the United States Medical Licensing Examination (USMLE) to make decisions in recruitment of potential applicants. This study was done to determine a correlation of USMLE Steps 1, 2 and 3 results with ITE scores in each level of Internal Medicine training. A retrospective review of all residents graduating from an Internal Medicine program from 1999 to 2006 was done. Subjects included had data for all USMLE Steps and ITE during all years of training. Thirty-one subjects were included in the study. Correlations of USMLE Steps 1, 2 and 3 were done with ITE scores (percent correct) in each year of training. Pearson's correlation coefficient (r) was determined for each pairing and a t test to determine statistical significance of the correlation was done. Statistical significance was defined as P value <0.05. The r values for USMLE Step 1 and ITE percent correct in PGY I, II and III were 0.46, 0.55 and 0.51 respectively. Corresponding r values for USMLE Step 2 and ITE percent correct were 0.79, 0.70 and 0.72; for USMLE Step 3 these values were 0.51, 0.37 and 0.51 respectively for each training year. USMLE scores are correlated with ITE scores. This correlation was strongest for USMLE Step 2.

show abstract

Internal medicine residents do not accurately assess their medical knowledge

Cited by 17 publications

References 18 publications

Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination

Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination

Understanding self-assessment as an informed process: residents’ use of external information for self-assessment of performance in simulated resuscitations

Correlation of United States Medical Licensing Examination and Internal Medicine In-Training Examination performance

Contact Info

Product

Resources

About