An Analysis of the Dimensionality of Toefl Reading Comprehension Items

Abstract

The issue of exactly what is measured by different types of reading items has been a matter of interest in the field of reading research for many years. Language teaching and testing specialists have raised the question of whether a reading test for foreign students wishing to enter university in the United States should include questions testing abilities beyond linguistic and very general discourse competencies, or indeed whether it is possible to separate these language competencies from other competencies. The purpose of this study was to investigate the dimensionality of the TOEFL® reading test, based on the specifications in use as of April 1991. Of particular interest was whether four item types identified in the test specifications as “reasoning items” could be shown to measure, in addition to general reading ability, any abilities not measured by the other item types in the TOEFL reading test. Two techniques, Stout's procedure and NOHARM analyses, were employed to investigate the hypothesized two‐factor model. In both cases the data failed to fit the model, indicating that TOEFL “reasoning items” cannot be shown to measure a unique construct. However, the follow‐up exploratory analyses indicated that all 10 test forms used in the study violated the assumption of essential unidimensionality, and all of the forms appeared to fit a two‐factor model where the second factor may be related to passage content or position.

FACTOR STRUCTURE OF THE TOEFL® INTERNET‐BASED TEST (IBT): EXPLORATION IN A FIELD TRIAL SAMPLE

“…The finding that these new item types are not psychometrically distinct from other items in the section was similar to the results of three previous studies of the TOEFL test (Hale et al, 1988;Manning, 1987;Schedl et al, 1996). The primary interest of these studies was how new Manning's findings agree in that the new item types studied by them had high degrees of overlap with the constructs assessed in the Structure and Written Expression and the Vocabulary and Reading Comprehension sections.…”

“…Moreover, across all three models, the patterns observed in the factor loadings were similar: moderate positive trait factor loadings, low and positive method factor loadings, and high inter-trait-factor correlations. These patterns observed in the trait factor loadings and correlations were substantively reasonable, considering the expected psychometrically unidimensional nature of the reading ability assessed in the TOEFL Reading section (Hale et al, 1988(Hale et al, , 1989Schedl, Gordon, & Tang, 1996;Swinton & Powers, 1980). The goodness of fit of these models was satisfactory as well.…”

An Investigation into the Dimensionality of TOEFL Using Conditional Covariance‐Based Nonparametric Approach

“…Research has been conducted to uncover the latent structure of the TOEFL and the relationship between dimensionality of the TOEFL and examinee characteristics, such as English proficiency, language background, ethnicity, and gender (Oller, 1979; Oltman & Stricker, 1990; Schedl, Gordon, Carey, & Tang, 1996). Dunbar's (1982) confirmatory factor analysis results suggested one dominant factor for different language groups along with various factors associated with testlets.…”

A cognitive processing model of reading comprehension in English as a foreign language using the linear logistic test model

“…Alderson (2005), based on the results obtained from the piloting of different DIALANG components, concluded that subskills do not contribute to distinguishing between different levels of reading language proficiency hence they are inseparable. Schedl, Gordon, Carey, and Tang (1996) studied the dimensionality of the reading section of the TOEFL (Test of English as Foreign Language) and concluded that different items which were supposed to measure different reading subskills do not explain differences in item difficulty or test dimensionality.…”