Predicting grades from an English language assessment: The importance of peeling the onion

Abstract

Data from 787 international undergraduate students at an urban university in the United States were used to demonstrate the importance of separating a sample into meaningful subgroups in order to demonstrate the ability of an English language assessment to predict the first-year grade point average (GPA). For example, when all students were pooled in a single analysis, the correlation of scores from the Test of English as a Foreign Language (TOEFL) with GPA was .18; in a subsample of engineering students from China, the correlation with GPA was .58, or .77 when corrected for range restriction. Similarly, the corrected correlation of the TOEFL Reading score with GPA for Chinese business students changed dramatically (from .01 to .36) when students with an extreme discrepancy between their receptive (reading/listening) and productive (speaking/writing) scores were trimmed from the sample.

Investigating the Predictive Validity ofTOEFL iBT®Test Scores and Their Use in Informing Policy in a United Kingdom University Setting

“…In line with findings reported in Cho and Bridgeman () and Bridgeman et al (), we found small but significant correlations for all TOEFL iBT scores for the total sample. Following Bridgeman et al's approach, we found that “peeling the onion” and looking into different subgroups for correlation patterns yielded a more nuanced picture.…”

“…Grouping students by nationality also yielded interesting results: The Chinese subgroup showed the strongest correlations, significant for speaking and overall scores, while for the Indian subgroup, the correlations were only significant for listening scores, broadly in line with the findings reported by Bridgeman et al (). In addition to these two groups, we also examined the subgroup of German students, who showed the smallest correlations, none of which was significant.…”

“…Interestingly, like Bridgeman et al (), we found that Chinese students exhibit a slightly different TOEFL iBT profile than the rest of the population. However, in our sample, it was not as pronounced as what Bridgeman et al reported so that we did not exclude any students on these grounds.…”

“…We first address RQ1, the relation between the language skills reported by the TOEFL iBT scores and students' subsequent academic performance as expressed in final academic grades. With regard to the question whether different subgroups of students show differing profiles in their TOEFL iBT section scores, like Bridgeman et al (), we found that Chinese students exhibit a slightly different profile of TOEFL iBT section scores from the rest of the population, yet it was much less pronounced than what Bridgeman et al reported. Hence we found no grounds upon which to recommend paying specific attention to certain groups for conspicuous profiles.…”

“…They found moderate yet meaningful correlations between the two indicators. Addressing the well‐known issue that heterogeneous groups can veil correlation patterns existing for more homogeneous subgroups, Bridgeman, Cho, and DiPietro () found higher correlations between TOEFL scores and GPA when grouping students by nationality and department as compared to the overall sample.…”