A Description of What Happens When an Examinee Takes a Multiple‐Choice Reading Comprehension Test

Abstract

This study investigated the strategies a group of college students used to complete a portion of a standardized reading comprehension test. Twenty‐six students were randomly assigned to either an introspective interview, in which the subjects explained to a researcher what they were doing and thinking as they read the test passages and answered the multiple‐choice questions, or a retrospective interview, in which the students completed the test without interruption and then recounted for the researcher how they had gone about the task. Data analysis resulted in the identification of three broad categories of processing behavior: an overall approach to the test task, reading strategies, and test‐taking strategies. In addition, difficulties encountered by the subjects were identified. Results indicate that the common element in each subject's approach to the test was a focus on getting to the questions as quickly as possible and then using the questions to direct a search of the passage to locate the best possible information to answer the questions. The implications of these results for better understanding the relationship between test‐taking behavior and reading are discussed.

Using working memory theory to investigate the construct validity of multiple-choice reading comprehension tests such as the SAT.

“…The four strategies were based on the ones identified by Farr et al (1990). For Strategy 1 (passage-first; entire passage), participants read the entire passage first.…”

“…The first and second strategies identified by Farr et al (1990) were passage-first strategies in that students consulted the passage before turning to the questions. The two strategies differed in the extent of the initial interaction with the passage.…”

“…With respect to the construct validity question, the four strategies identified by Farr et al (1990) all involved consulting or searching local parts of the passage to find information to answer a specific question. However, there were differences among the four strategies in how much global reading of a passage took place before the test taker initiated question-directed search of the passage.…”

“…Experiment 1 Farr et al (1990) identified four general strategies that test takers spontaneously use when completing a multiple-choice reading comprehension test. However, Farr et al did not analyze test success as a function of the strategy used, nor did they investigate the extent to which test-taking strategy affected the validity of the test as a measure of passage comprehension.…”

“…In particular, we use the working memory approach as a way to investigate the construct validity of the reading comprehension portion of the revised Scholastic Assessment Test (SAT). There has been a long and persistent history of attacks on the validity of multiple-choice tests of reading comprehension such as the SAT (see Anderson, Hiebert, Scott, & Wilkinson, 1985;Cohen, 1984;Drum, Calfee, & Cook, 1981;Farr, Pritchard, & Smitten, 1990;Katz, Blackburn, & Lautenschlager, 1991;Katz, Lautenschlager, Blackburn, & Harris, 1990;Royer, 1990). One of the most serious criticisms is that test takers do not or need not read and comprehend the passages on which the test questions are based.…”