Background. A two-step selection process, consisting of cognitive and noncognitive measures, is common in medical school admissions. Objective. To estimate the validity of this process in predicting academic performance, taking into account the complex and pervasive effect of range restriction in this context. Methods. The estimation of the validity of the two-step process included a sequential correction for range restriction and an estimation of the predictive validity of the process in its entirety. Data were collected from 1,002 undergraduate students from four cohorts (2006/07-2009/10) at three medical schools in Israel. Results. The predictive validity of the composite of the cognitive measures with respect to Year 1 performance was high, resulting entirely from the predictive validity of the admission test (a standard measure of ability). The predictive validity of the noncognitive measure was moderate. The predictive validity of the process in its entirety was high, its value dependent on the weights given to the cognitive and noncognitive measures. Conclusion. A cognitive admission test has a high predictive validity with respect to Year 1 performance. The addition of a noncognitive measure in the second step does not markedly diminish the predictive validity of the selection process with respect to academic achievement.

The study provides important evidence for the validity of the MMI. Additional studies are required to reevaluate the predicted validity of the MMI.

No abstract

A frequent topic of psychological research is the estimation of the correlation between two variables from a sample that underwent a selection process based on a third variable. Due to indirect range restriction, the sample correlation is a biased estimator of the population correlation, and a correction formula is used. In the past, bootstrap standard error and confidence intervals for the corrected correlations were examined with normal data. The present study proposes a large-sample estimate (an analytic method) for the standard error, and a corresponding confidence interval for the corrected correlation. Monte Carlo simulation studies involving both normal and non-normal data were conducted to examine the empirical performance of the bootstrap and analytic methods. Results indicated that with both normal and non-normal data, the bootstrap standard error and confidence interval were generally accurate across simulation conditions (restricted sample size, selection ratio, and population correlations) and outperformed estimates of the analytic method. However, with certain combinations of distribution type and model conditions, the analytic method has an advantage, offering reasonable estimates of the standard error and confidence interval without resorting to the bootstrap procedure's computer-intensive approach. We provide SAS code for the simulation studies.

No abstract