Can Admissions Interviews Predict Performance in Residency?

Abstract

As they are currently conducted, application interviews do not have sufficient power to predict performance during residency. Letters of reference may be useful to the extent that they reflect personal experience with the applicant, but differences in ratings of these letters are not great enough to base admission decisions on them. As it is currently performed, the interview process may be more useful as a means of interesting applicants in the program than of evaluating their potential for success in the residency.

Use of the Interview in Resident Candidate Selection: A Review of the Literature

“…Of the 34 studies (TABLE 3), 17 showed that the interview did not predict subsequent clinical performance (weak positive, negative, or no correlation) in internship or residency, particularly with a traditional or unstructured interview format. 41,50,[53][54][55][56][57][58][59][60][61][62][63][64][65][66][67] Eleven studies showed a positive correlation between interview and subsequent performance. 8,51,52,[68][69][70][71][72][73][74][75] Performance metrics with a positive correlation included clinical evaluations, in-training examinations, licensing board examinations, and a composite score or rank of resident performance.…”

“…Attrition from residency was not consistently predicted by the interview process in the 6 studies that assessed this variable. [4][5][6]43,55,56 In a case control study of psychiatry residents, minor or major problems during residency may have only been associated with negative comments in the dean's letter and were not picked up in the interview. 76 A longitudinal study from 1965 to 1999 showed that neither the residency interview nor the performance evaluations in a psychiatry program predicted longterm professionalism issues, resulting in referral to an impaired physician program.…”

Can We Predict “Problem Residents”?

“…More recently, the same group examined the relationship between admission interviews, letters of reference, and residency performance in 544 residents across 41 years. 17 They found some modest correlations but concluded that the differences were not great enough to be of practical utility.…”

Comparative Reliability of Structured Versus Unstructured Interviews in the Admission Process of a Residency Program

“…4,[12][13][14][15][16][17] Studies of the predictive validity of interviews for resident clinical performance have found mixed results. [18][19][20][21][22] The interviews in these studies were not structured, and only one study 21 reports an interrater reliability (another study 18 describes the interviews as structured, but the description does not support this statement). Although the predictive validity of interviews is the ultimate goal, their reliability has to be optimized first before they can be used in a predictive manner.…”