Bias, Fairness, and Validity in Graduate-School Admissions: A Psychometric Perspective

Abstract

As many schools and departments are considering the removal of the Graduate Record Examination (GRE) from their graduate-school admission processes to enhance equity and diversity in higher education, controversies arise. From a psychometric perspective, we see a critical need for clarifying the meanings of measurement “bias” and “fairness” to create common ground for constructive discussions within the field of psychology, higher education, and beyond. We critically evaluate six major sources of information that are widely used to help inform graduate-school admissions decisions: grade point average, personal statements, resumes/curriculum vitae, letters of recommendation, interviews, and GRE. We review empirical research evidence available to date on the validity, bias, and fairness issues associated with each of these admission measures and identify potential issues that have been overlooked in the literature. We conclude by suggesting several directions for practical steps to improve the current admissions decisions and highlighting areas in which future research would be beneficial.

The ACT Predicts Academic Performance—But Why?

“…As outlined in the Standards for Educational and Psychological Testing ( Joint Committee 2018 ), fairness is a broad concept; we use the term in this context mainly to refer to issues of equity, such as differential access to scholarly opportunities, and the consequences of these inequities. For a review, see Woo et al ( 2022 ).…”

Constructs, Tape Measures, and Mercury

“…It would be racist to think that experiences with White people improved the latent ability within Black people, but I do concede that experiences with White people and White culture may help Black students navigate graduate school in predominantly White universities. However, the latent constructs defined by the GRE and discussed by Woo et al (2022) are not defined with this kind of flexibility of meaning, and I, among others, argue that they should be (Sireci, 2021). It is a choice, an irresponsible one, to define these constructs as measures of ability with meaning that generalizes across groups.…”

“…Squaring the coefficients-because we do not know the causal direction (only the temporal one, given that both GRE scores and grades may be indicators of privilege taken at two time points)-yields the knowledge of a 3% to 15% known variance space about what is supposed to be a measure of ability because it predicts first-year grades but is a better predictor of race. Therefore, most of what is predicting first-year grades is unknown, and we have no idea whether removing bias from predicting 3% to 15% of the variance space generalizes to the other 85% to 97% (Cohen et al, 2003;Sternberg & Williams, 1997;Woo et al, 2022). The correlation between individuals' height and their tape measured height is 1 to 1 (100%, for a coefficient of 1.00), thus we can confidently audit these models for biased prediction across groups.…”

“…In addition, they critically examine the sometimes-limited empirical evidence about bias and fairness in six types of information used to evaluate students during the graduate-school admissions process, including grade point average, personal statements, resumes/curriculum vitae (CVs), letters of recommendation, interviews, and the GRE. I believe that within a world constrained by commonly accepted psychometric approaches to thinking about bias and fairness-a world that has perpetuated the standardized testing industry-this examination was well executed by Woo et al (2022). Their review of bias and fairness in prediction regarding the GRE and other materials evaluated for graduate-student admissions demonstrates a wealth of information worth reviewing.…”

Sesgo en las Notas de Enseñanza Media (NEM): propuestas para perfeccionar los instrumentos de selección a la educación superior

“…La capacidad predictiva es un factor relevante al momento de evaluar los instrumentos de selección. Eliminar los sesgos de los instrumentos no cumple su objetivo si los estudiantes que logran acceder a una mejor educación universitaria gracias a ello no logran tener éxito en esta educación (Woo et al 2022;Baldwin y James 2010;Camara 2005;Perffeto 1999;Santelices 2007).…”