The Predictive Validity of a Two-Step Selection Process to Medical Schools

Abstract

Background. A two-step selection process, consisting of cognitive and noncognitive measures, is common in medical school admissions. Objective. To estimate the validity of this process in predicting academic performance, taking into account the complex and pervasive effect of range restriction in this context. Methods. The estimation of the validity of the two-step process included a sequential correction for range restriction and an estimation of the predictive validity of the process in its entirety. Data were collected from 1,002 undergraduate students from four cohorts (2006/07-2009/10) at three medical schools in Israel. Results. The predictive validity of the composite of the cognitive measures with respect to Year 1 performance was high, resulting entirely from the predictive validity of the admission test (a standard measure of ability). The predictive validity of the noncognitive measure was moderate. The predictive validity of the process in its entirety was high, its value dependent on the weights given to the cognitive and noncognitive measures. Conclusion. A cognitive admission test has a high predictive validity with respect to Year 1 performance. The addition of a noncognitive measure in the second step does not markedly diminish the predictive validity of the selection process with respect to academic achievement.

Academic Achievements of College Graduates With Learning Disabilities Vis-a-Vis Admission Criteria and Academic Support

“…Individuals with LD face barriers in entering into postsecondary education programs, which raises questions regarding the adequacy of the college admission criteria for this population. The admission procedure to colleges and universities includes standardized norm-referenced entrance examinations such as the Scholastic Aptitude Test (SAT) in the United States (Zurcher & Bryant, 2001) and the Psychometric Test in Israel (Kennet-Cohen, 2001). Research in U.S. institutions has shown that scores on entrance examinations to postsecondary education programs for students with LD have a low correlation with criteria such as BA grade point average (GPA) even when accommodations are provided (Zurcher & Bryant, 2001).…”

“…The main criteria for selecting candidates for undergraduate studies in universities and colleges in Israel are the applicants’ score on the matriculation exam (a state-administered test that students usually take after completing 12 years of education) and their score on a psychometric exam. A composite score is calculated, usually by ascribing equal weight to each exam (Haimowitz, Ben-Shakhar, 2004; Kennet-Cohen, 2001). Applicants to colleges who have satisfactory matriculation grades, but not necessarily a psychometric entrance exam score, are accepted conditionally for 1 year, after which they can continue if their mean GPA meets the requirements of the college.…”

“…The use of the psychometric exam as an entrance requirement for higher education has become an issue of public debate in Israel due to the disparity it creates between social groups (Gamliel & Cahan, 2004). On one hand, psychometric exam scores have high criterion validity, predicting GPA university scores (Beller, 2001; Kennet-Cohen, 2001; Kleper, Turvall, & Oren, 2014). However, a recent study (Ben-David & Shahor, 2012) that focused on college students undermined the validity of the psychometric score as a predictor of GPA.…”

Standard errors and confidence intervals for correlations corrected for indirect range restriction: A simulation study comparing analytic and bootstrap methods

“…In such circumstances, the accumulation of time saved may matter. The analytic method was recently applied in such circumstances to an examination of the validity of medical school admissions processes (Kennet-Cohen, Turvall, Saar, & Oren, 2016).…”