FACTOR STRUCTURE OF THE TOEFL^® INTERNET‐BASED TEST (IBT): EXPLORATION IN A FIELD TRIAL SAMPLE

Abstract

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet‐based test (TOEFL® iBT). An item‐level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003–2004 TOEFL iBT Field Study. CFA‐based multitrait‐multimethod (MTMM) analyses for the Reading and Listening sections showed that the language abilities assessed in each section were essentially unidimensional, while the factor structure of the entire test was best represented by a higher‐order factor model with a general factor (English as a second language/English as a foreign language ability) and four group factors for reading, listening, speaking, and writing. The integrated Speaking and Writing tasks, which require language processing in multiple modalities, well defined the target modalities (speaking and writing). These results broadly support the current reporting of four scores corresponding to the modalities and a total score, as well as the test design where the integrated tasks contribute only to the scores for the target modalities.

INVESTIGATING THE VALUE OF SECTION SCORES FOR THE TOEFL iBT® TEST

“…Meanwhile, the results supporting the distinctiveness of the constructs assessed in the reading, listening, and writing sections are mixed. Many issues might have contributed to the difference of the findings between the Stricker et al study versus Sawaki et al (2008) and Stricker and Rock (2008) on this issue. One potential reason has to do with the differences in the test design.…”

“…In their confirmatory factor analysis (CFA) using item parcels, they identified two correlated factors, one for a fusion of reading, listening, and writing sections and the other for the speaking section. Sawaki, Stricker, and Oranje (2008) conducted an item-level factor analysis of a TOEFL iBT field study test form. In this study a higher order factor model with a general factor for English as a second language or English as a foreign language (ESL/EFL) ability and four first-order factors corresponding to the four modalities were identified.…”

“…This is primarily because the present study was the first factor analysis of operational TOEFL iBT data. The previous factor analyses were conducted on a TOEFL iBT prototype by Stricker et al (2005) and on field study data by Sawaki et al (2008) and Stricker and Rock (2008). Following the suggestions in the literature to use different data sets for exploring the factor structure and confirming the findings of the exploratory analyses (e.g., Jöreskog, 2007, p. 58), EFAs were conducted on two randomly selected forms (the April and December forms), and CFAs were conducted on the other two forms (the July and September forms).…”

“…Relative goodness of fit of four CFA models was examined to choose the best among four proposed models for each sample. The models were constructed based on those tested by Sawaki et al (2008), Stricker and Rock (2008), and Stricker et al (2005).…”

“…The correlated four-factor model allows examination of the relationships across the four modalities but not their relationships to general language ability. A higher order factor model was identified as the final model by a recent study of TOEFL iBT field study data by Sawaki et al (2008). Stricker and Rock (2008) adopted a higher order factor model as well, although their final model did not include the cross-loadings.…”