Using third-party evaluations to assess socioemotional skills in graduate and professional school admissions

Abstract

Consideration of socioemotional skills in admissions potentially can increase representation of racial and ethnic minorities and women in graduate education as well as identify candidates more likely to succeed in graduate school. Research on one such assessment, the ETS Personal Potential Index (PPI), showed that the PPI produced much smaller racial/ethnic-gender group mean score differences than undergraduate grade point average (UGPA) and the Graduate Record Examinations (GRE) did. Across levels of institutional selectivity, the PPI can promote racial/ethnic and gender diversity in graduate and professional school in ways that UGPA and GRE scores do not. Predictive validity analyses showed that for doctoral STEM programs the PPI dimensions of (i) Planning and Organization and (ii) Communication Skills positively predict graduate school grade point average as well as a lower risk of academic probation, a determinant of degree progress, both alone and incrementally over UGPA and GRE scores.

Studies of Possible Effects of GRE® ScoreSelect® on Subgroup Differences in GRE® General Test Scores

“…Empirical evidence shows that mean initial GRE scores are often higher for Whites, males, U.S. citizens, those under age 26, and those who have one or both parents or guardians who attended at least some graduate school. However, Chinese citizens, non‐U.S., non‐Chinese citizens, and Asian Americans score higher on average on GRE‐Q than do comparison groups (ETS, 2013; Gallagher et al, 2000; Klieger et al, 2022), and older examinees score higher on average on GRE‐V than do younger examinees (ETS, 2013; Trapani, 2013). The latter finding is consistent with research on the effects of aging on cognition, which shows that at least some verbal abilities increase into middle and later adulthood (see, e.g., Salthouse, 2004).…”

Holistic Admissions in Higher Education

“…Research shows that interrater reliability across three raters often fails to exceed .40 (Baxter et al, 1981). Such low reliability limits the predictive validity of reference letters (Klieger et al, 2022). Even quality letters of recommendation provide only modest prediction of graduate school outcomes such as GPA and faculty ratings (Kuncel et al, 2014).…”

“…Given the open-endedness in reference letters and idiosyncrasies in interpretation of letters, efforts have been made to standardize reference letters for better clarity and improved equity in admissions (Klieger et al, 2022;Liu et al, 2009). For example, Klieger and coauthors reported the use of a standardized tool for letters of recommendation, called the Personal Potential Index (PPI), in graduate school admissions.…”