Relationships Among Classical Test Theory and Item Response Theory Frameworks via Factor Analytic Models

Abstract

There are well-defined theoretical differences between the classical test theory (CTT) and item response theory (IRT) frameworks. It is understood that in the CTT framework, person and item statistics are test- and sample-dependent. This is not the perception with IRT. For this reason, the IRT framework is considered to be theoretically superior to the CTT framework for the purpose of estimating person and item parameters. In previous simulation studies, IRT models were used both as generating and as fitting models. Hence, results favoring the IRT framework could be attributed to IRT being the data-generation framework. Moreover, previous studies only considered the traditional CTT framework for the comparison, yet there is considerable literature suggesting that it may be more appropriate to use CTT statistics based on an underlying normal variable (UNV) assumption. The current study relates the class of CTT-based models with the UNV assumption to that of IRT, using confirmatory factor analysis to delineate the connections. A small Monte Carlo study was carried out to assess the comparability between the item and person statistics obtained from the frameworks of IRT and CTT with UNV assumption. Results show the frameworks of IRT and CTT with UNV assumption to be quite comparable, with neither framework showing an advantage over the other.

On the Relationship Between Classical Test Theory and Item Response Theory

“…These relationships can be seen as closely tied to those between factor analysis (FA) and IRT, which have been pointed out in the methodological literature over the past 30 years or so (e.g., Kamata & Bauer, 2008;B. O. Muthén, Kao, & Burstein, 1991;Takane & de Leeuw, 1987; see also Kohli, Koran, & Henn, 2014). The CTT-IRT relationships, when appropriately highlighted and methodically clarified for empirical educational, behavioral, and social researchers, can in our view significantly contribute to improvements in their measurement related work as well as to tangible progress in the entire field of measurement.…”

Measuring Sense of Community Responsibility in Community‐Based Prevention Coalitions: An Item Response Theory Analysis

“…Unlike factor analysis, however, IRT assumes nominal or ordinal indicator variables, which entails different modeling and estimation methods. Although several researchers have shown that CTT and IRT often produce similar conclusions concerning the psychometric properties of measures (e.g., Kohli, Koran, & Henn, ; Raykov & Marcoulides, ), it is useful to conduct IRT because its results provide a distinct way of evaluating the psychometric properties of scales and their component items, and can demonstrate robustness of results obtained using CTT methods.…”

Estimating person parameters via item response model and simple sum score in small samples with few polytomous items: A simulation study

“…However, when skewed items were analyzed, the θ often outperformed the sum score. Various other studies compared the θ‐values with the corresponding sum score from real data as well as from simulated data and found similar minor differences. In an example using real data, various tests of extraversion and neuroticism were summarized via θ‐values and sum scores; the median correlations were r = .96.…”