BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice

Hamdy, Hossam; Prasad, Kameshwar; Anderson, M. Brownell; Scherpbier, Albert; Williams, Reed G.; Zwierstra, R. P.; Cuddihy, Helen

doi:10.1080/01421590600622723

Cited by 158 publications

(131 citation statements)

References 62 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…[2][3][4][5][6] However, the relationship between commonly available measures of academic performance and future competency is largely unknown. 2 Andriole et al found that medical school grades and USMLE exam scores predicted program director ratings of surgical interns. 7 Knowledge of which markers predict poor performance during internship would be useful to medical school admission committees, medical educators, and intern program directors.…”

Section: Introductionmentioning

confidence: 99%

“…2 For example, standardized tests tend to predict future standardized test scores and grades earned at 1 educational level tend to predict grades earned at subsequent training levels. [2][3][4][5][6] However, the relationship between commonly available measures of academic performance and future competency is largely unknown. 2 Andriole et al found that medical school grades and USMLE exam scores predicted program director ratings of surgical interns.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identifying Medical Students Likely to Exhibit Poor Professionalism and Knowledge During Internship

et al. 2007

View full text Add to dashboard Cite

PARTICIPANTS:One thousand sixty-nine graduates between 1993 and 2002. MAIN OUTCOME MEASURE(S):Residency program directors completed an 18-item survey assessing intern performance. Factor analysis of these items collapsed to 2 domains: knowledge and professionalism. These domains were scored and performance dichotomized at the 10th percentile. RESULTS:Many variables showed a univariate relationship with ratings in the bottom 10% of both domains. Multivariable logistic regression modeling revealed that grades earned during the third year predicted low ratings in both knowledge (odds ratio [OR]=4.9; 95%CI=2.7-9.2) and professionalism (OR= 7.3; 95%CI=4.1-13.0). USMLE step 1 scores (OR=1.03; 95%CI=1.01-1.05) predicted knowledge but not professionalism. The remaining variables were not independently predictive of performance ratings. The predictive ability for the knowledge and professionalism models was modest (respective area under ROC curves=0.735 and 0.725). CONCLUSIONS:A strong association exists between the third year GPA and internship ratings by program directors in professionalism and knowledge. In combination with third year grades, either the USMLE step 1 or step 2 scores predict poor knowledge ratings. Despite a wealth of available markers and a large data set, predicting poor performance during internship remains difficult.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Identifying Medical Students Likely to Exhibit Poor Professionalism and Knowledge During Internship

et al. 2007

View full text Add to dashboard Cite

show abstract

“…But of the 9 published BEME reviews, only one was able to estimate an overall effect. Ironically, this was a review of predictive validity of assessment methods, based on correlational studies (Hamdy et al 2006). But no one would suggest that we abandon educational experiments although a few people (Regehr 2010;Mennin 2010), have suggested, or at least implied, just that.…”

mentioning

confidence: 99%

Is experimental research passé

Norman

2010

Adv in Health Sci Educ

View full text Add to dashboard Cite

In perusing the contents of this issue, I discovered that, once again, the majority of the papers are not experimental. There are several qualitative studies-of teachers' attitudes to student feedback, reflective learning for continuing education, residents' perceptions of outpatient teaching, and assessment expert's view of the framework for assessments, cohort study looking at predictors of motivation for medicine, a survey of teachers' perceptions of student feedback, a correlational study of concurrent validity of a personality test, a review paper on the issue of whether Asian students are rote learners, and 3 randomized trials. One looked at the use of simulation in addition to a PBL session for management of respiratory and cardiac distress, another at web-based learning, and a third examined the effect of testing on retention of CPR skills. So of the 11 original studies, 3 were experimental.I suppose I should be pleased. Todres et al. (2007), in a widely cited review of 2 years of Medical Education and Medical Teacher articles, found only 5% were randomized controlled trials. So at 3/11 = 27%, we're doing much better.Better? Pardon a second. Should we be striving for 100% randomized trials? Is this any measure of quality of research? After all, a study of faculty perception of student feedback is unlikely to benefit from having a control group (randomized of course) who were not given feedback and then asked their perceptions of not getting feedback. For that matter, randomizing students to be Asian or not is a bit impractical. In short, the variety of research designs in the papers in this issue reflect the nature of the research questions asked, which really span the gamut of educational research, and only a few are amenable to experimentation.But what of quality? One carryover from clinical research is a perverse hierarchy of research designs where randomized controlled trial is at the top in quality, case series is at the bottom, and cohort studies are somewhere in the middle. The first problem with this approach is that it is a very limited taxonomy, suitable for epidemiologic studies and little else. The common approach to a study of assessment, looking at reliability and validity,

show abstract

“…31 In summary, these findings highlight a potentially negative impact of breaking from the traditional curriculum. Our study is limited by several factors.…”

Section: Discussionmentioning

confidence: 99%

“…In a recent BEME systematic review of predictive values of medical school performance and future performance, clerkship honors and grades correlated positively with supervisor rating during residency. 31 Similarly, Taylor et al reported that medical school GPA was the best predictor of residency directors' ratings of interns. 32 Others have found clerkship honors to predict performance on the American Board of Specialty Examination 33 along with residency performance.…”

Section: Discussionmentioning

confidence: 99%

Prolonged Delays for Research Training in Medical School are Associated with Poorer Subsequent Clinical Knowledge

et al. 2007

View full text Add to dashboard Cite

BACKGROUND:Complementary degree programs and research training are important alternative tracks in medical school that typically interrupt the traditional MD curriculum. OBJECTIVE:Examine effects of such a break on clinical knowledge after reentry into the MD curriculum. DESIGN:Retrospective cohort study. PARTICIPANTS:Three hundred and two graduates of Mayo Medical School. MAIN MEASUREMENTS:Compared years of delay between the second and third years of medical school with third year clerkship grades, National Board of Medical Examiner's (NBME) Subject Examinations, and United States Medical License Exam (USMLE)Step 2.MAIN RESULTS: 258, 13, and 31 students spent 0, 1, or ≥3 years pursing research between the second and third year. Baseline measures of knowledge before matriculation and before the third year were similar between groups. Whereas a 1-year delay had no significant effect, a ≥3-year delay was associated with fewer clerkship honors and lower NBME Medicine, Pediatrics, and Psychiatry percentiles compared to no delay (all p<.05). Students with a ≥3-year delay had a 77% reduction in the odds of honors in Medicine. For each year of delay beyond 3, students' third-year NBME Medicine, Neurology, Obstetrics and Gynecology, and Psychiatry scores decreased as did USMLE Step 2 scores (r=−.38 to −.50, p<.05).CONCLUSIONS: Delays of ≥3 years between the second and third years of medical school are associated with lower grades and scores on clinical knowledge tests. Further research is needed to determine the optimal timing of research training and develop effective interventions to facilitate reentry into the medical school curriculum.

show abstract

BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice

Cited by 158 publications

References 62 publications

Identifying Medical Students Likely to Exhibit Poor Professionalism and Knowledge During Internship

Identifying Medical Students Likely to Exhibit Poor Professionalism and Knowledge During Internship

Is experimental research passé

Prolonged Delays for Research Training in Medical School are Associated with Poorer Subsequent Clinical Knowledge

Contact Info

Product

Resources

About