Component Latent Trait Models for Paragraph Comprehension Tests

Abstract

The cognitive characteristics of paragraph comprehension items were studied by comparing models that deal with two general processing stages: text representation and response decision. The models that were compared included the prepositional structure of the text (Kintsch & van Dijk, 1978), various counts of surface structure variables and word frequency (Drum et al., 1981), a taxonomy of levels of text questions (Anderson, 1972), and some new models that combine features of these models. Calibrations from the linear logistic latent trait model allowed evaluation of the impact of the cognitive variables on item responses. The results indicate that successful prediction of item difficulty is obtained from models with wide representation of both text and decision processing. This suggests that items can be screened for processing difficulty prior to being administered to examinees. However, the results also have important implications for test validity in that the two processing stages involve two different ability dimensions. Susan E. Whitely. This work was supported by the Navy Personnel Research and Development Center through the Army Research Office and Battelle Memorial Institute Contract No. 0855. The opinions expressed in this article are those of the authors, are not official, and do not reflect the views of the Departments of the Navy or Army.

Effects of Dynamic Assessment on the Acquisition of the Rhythm of English: The Case of EFL Learners’ Attitudes

“…More specifically, DA focuses on the evaluation process as well as the product. It attempts to modify the student's performance during testing by introducing material or instructions to elicit higher achievement levels (Embretson & Wetzel, 1987).…”

The Prediction of Gre Reading Comprehension Item Difficulty for Expository Prose Passages for Each of Three Item Types: Main Ideas, Inferences and Explicit Statements

“…Only a few studies appear to have focused on predicting item difficulty using items from standardized ability tests (Drum, Calfee, & Cook, 1981;Embretson & Wetzel, 1987). While not specifically focused on predicting reading item difficulty, many other studies of language processing have isolated a wide variety of variables that influence comprehension difficulty with respect to decision time and recall measures.…”

“…They reported good predictability using these simple surface variables; on average, they indicated that about 70% of the variance 1 of multiple-choice reading item difficulty was explained. Embretson and Wetzel (1987) also studied the predictability of 75 reading item difficulties using a few of the surface variables studied by Drum et al (1981). But in addition, because of the brevity of their passages, Embretson and Wetzel were able to do a propositional analysis (see Kintsch & van Dijk, 1978) and add variables from this analysis, along with several other measures, as predictor variables.…”

“…Instead many used dependent measures such as recall of passages or decision time to infer the influence that certain variables have on comprehension difficulty. This review along with our earlier review of the Drum et al (1981) and Embretson &Wetzel (1987) studies will help us to select a final set of variables that we postulate may also index comprehension difficulty within a multiple-choice testing format. Carpenter and Just (1975) found that the occurrence of sentence negation increases comprehension decision time.…”

Toward a Cognitive Basis for Quantitative Ability Measures

“…Secondly, different problem attributes have differential effects on students' strategies and errors. Thus, construct validity problems might arise when examinees are tested on different sets of problems, as in computer adaptive testing (Embretson & Wetzel, 1987). If one examinee receives problems that are difficult because of the need to formulate an algebraic representation while another examinee is tested on problems that are difficult because of structural complexity, the same abilities may not be brought to bear by both examinees.…”