The Development, Investigation, and Evaluation of New Item Types for the Gre Analytical Measure

Self Cite

A previous study of new item types for the analytical measure of the GRE General Test found that the items loaded on three of four separable factors that were labeled verbal reasoning, informal reasoning, formal-deductive reasoning, and quantitative reasoning. The present study examined the issue of how processing differed for these item types in the context of a problemspace framework. Protocols of examinees solving a small set of problems aloud were collected. These protocols were examined with respect to two phases of the problem-solving process: problem representation and problem solution. For formal-deductive items, all the information necessary to solve the probJem was provided in the problem statement. The representation of formal-deductive items involved the use of meaning-reduced tokens and spatial diagrams. The units involved in the representation of informal reasoning and verbal reasoning item types included meaningful propositions and meaning-emphasizing paraphrases. Reference to common background knowledge occurred. The analysis of the problem-solution phase focused on the processes of evaluation, (judgments of the cczreccness of an option) and justification (statements of an argument or of evidence for why an option was or was not correct). First, the order of these processes was found to differ for formal-deductive items and other item types. Secondly, item solutions varied in terms of the kinds of justifications that were offered by the examinees for accepting or rejecting options. These results illustrate how the addition of some item types to the GRE analytical measure will expand the variety of reasoning skills assessed. Implications of these results for cognitive models of reasoning are also discussed.Acknowledgments

Section: Summary and Discussionmentioning

confidence: 99%

Section: Background and Related Research On The Gre Analytical Measurementioning

confidence: 99%

Section: Analysis Of Explanations (Ax) (Revised) This Item Type Devmentioning

confidence: 99%

Section: Analysis Of Explanations (Ax) (Revised) This Item Type Devmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

A Cognitive Analysis of Solutions for Verbal, Informal, and Formal‐deductive Reasoning Problems

Enright

Tucker

Katz

1995

Self Cite

Development and Evaluation of Computer‐administered Analytical Questions for the Graduate Record Examinations General Test

Bridgeman

Rock

1992

Self Cite

Three new computer‐administered items types for the analytical scale of the Graduate Record Examination General Test were developed and evaluated. One item type was a free‐response version of the current analytical reasoning item type. The second item type was a somewhat constrained free‐response version of the pattern identification (or number series) item type in which the student had to state the rule that generated the series. The third item type used the computer to administer yes/no analysis of explanations questions with a limited branching strategy. The computer tests were administered at four ETS regional offices to a sample of students who had previously taken the GRE General Test. Scores from the regular GRE administration and the special computer administration were matched for a sample of 349 students. A number of test administration design issues were identified, including the need to provide adequate practice exercises, design of an interface comfortable for computer‐literate students, and problems with item‐level timing. The pattern identification items were too difficult (or the practice was inadequate), but the other items appeared to function well. There was no evidence that the open‐ended analytical reasoning items were measuring anything beyond what is measured by the current multiple‐choice version of these items.

Delimiting the Verbal Domain

DeMauro¹,

Merritt²,

Adams³

1994

A functional analysis was made of the verbal domain as it is defined by the test development process at Educational Testing Service. The thesis of the study is that the verbal domain consists of semantic relations that examinees must interpret. Test developers use a system of rules to sample stimulus materials from among infinite possibilities that elicit an interpretive response. Many of these rules are codified in a body of memoranda and corporate publications, and others are in different stages of formation, beginning in the use by individual test development professionals, and eventually becoming part of generally accepted practice. Because the process of test development is central to the formation of the verbal domain, a job analysis approach was used to examine how rules are employed to present the desired semantic relations to the examinees for their interpretation. This approach explains the process of delimiting the verbal domain through a comprehensive review of the uses of the rule system that defines when the items are functioning properly and when they are not. Test developers from two major divisions of ETS were observed and interviewed. They also rated the importance of each of the rules, suggested rules that they use in their work that were not listed, and delineated the professional activities for which these rules are most important. The study indicates that the rule system for sampling semantic relations is an evolving body, both for the individual test developer and for test development as an institution.