AN EXPERIMENT ON MULTIPLE‐CHOICE <i>VERSUS</i> OPEN‐ENDED ANSWERING IN A VOCABULARY TEST

Heim, A. W.; Watts, K. P.

doi:10.1111/j.2044-8279.1967.tb01950.x

Cited by 24 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inconsistent findings reresponse format. To control for the retention effect sulted in similar comparisons between the keylist inherent in a study by Heim and Watts (1967) who and multiple-choice completion formats. The keyused items measuring verbal skills, Traub and Fisher list format was relatively easy in terms of the peradministered items rewritten in the multiple-choice centage of possible points achieved by examinees, format two weeks later.…”

Section: Introductionmentioning

confidence: 99%

Comparison of Difficulties and Reliabilities of Quantitative Word Problems in Completion and Multiple-Choice Item Formats

Oosterhof

Coats

1984

Applied Psychological Measurement

View full text Add to dashboard Cite

Quantitative word problems were written as parallel completion and multiple-choice items, and were administered to 232 undergraduate students to compare the reliabilities and item difficulties associated with these formats. The multiple-choice options were written using specific numerical responses for each of five alternatives, revised by replacing the fifth option with "none of the above," and also by replacing each of the five responses with ranges of numerical values.Differences in distributions of scores imply a need to reestablish standards if changes are made in the proportions of completion and multiple-choice items included in a test. Findings did not support camouflaging the correct response by using "none of the above" or ranges of numerical values as multiple-choice alternatives. The increased time required to develop and administer a multiple-choice test with reliability equal to that of a completion test suggests use of the latter even in classes with relatively large enrollments.Authors of educational measurement texts generally favor use of test items that require making a choice among specified alternatives in contrast to items that require the examinee to produce a limited free response. Wesman (1971) recommended against the use of short-answer items, concluding that their superiority over selection-type items is more apparent than real in actual testing situations. Ebel (1979) indicated that short-answer items are used mainly to test for factual informa-APPLIED PSYCHOLOGICAL MEASUREMENT tion, and that good objective test items do not permit identification of the correct response on the basis of simple recognition or sheer rote memory. Popham (1981) took a more cautious approach by suggesting that a major weakness of multiple-choice items is the ability of examinees to recognize correct answers that, without assistance, they would not be able to construct.Instructors who develop classroom examinations that require students to provide a numerical response to a mathematical problem are often concerned about the appropriateness of the multiplechoice format. The present study augments previous research relevant to this concern by comparing the difficulty and reliability of multiple-choice and completion item formats as applied to the classroom measurement of quantitative skills. This investigation also included two variations of the multiple-choice format designed to reduce cues provided by alternatives. Focus is placed on the external validity of the experiment by using an actual examination of course material administered to students in a realistic classroom setting.Background

show abstract

Section: Introductionmentioning

confidence: 99%

Comparison of Difficulties and Reliabilities of Quantitative Word Problems in Completion and Multiple-Choice Item Formats

Oosterhof

Coats

1984

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…They question whether MC testing produces invalid results, or rather less conclusive results, than constructed-response exams. Bridgemann and Rock's (1992) study on college admittance testing, however, found otherwise; and other studies that compare essay exams and MC tests also show a high correlation between the two as student measurement tools (Bennet, Rock, & Wang, 1990;Bracht & Hopkins, 1970;Bridgeman & Rock, 1993;Heim & Watts, 1967;Joorabchi & Chawhan, 1975;Patterson, 1926;Traub & Fisher, 1977;Traub, 1993;Ward, 1982).…”

Section: Multiple-choice (Mc) Items Were Introduced In the Armymentioning

confidence: 99%

Test Item Construction and Validation: Developing a Statewide Assessment for Agricultural Science Education

Rivera¹

2011

Career and Technical Education Research

View full text Add to dashboard Cite

“…Early work comparing multiple-choice tests with constructed-response tests (Davis & Fifer, 1959;Heim & Wats, 1967;Vemon, 1962) generally indicated that tests employing different formats cannot be expected to have the same means, standard deviations, and correlations with criterion variables. Some of these differences can be assumed to be due to changes in the scale of measurement and amount of error variance associated with each format.…”

mentioning

confidence: 99%

A Comparison of the Information Provided by Essay, Multiple-Choice, and Free-Response Writing Tests

Ackerman

Smith

1988

Applied Psychological Measurement

View full text Add to dashboard Cite

This study investigated the similarity of information that is provided by direct and indirect methods of writing assessment. The skills required by each of these techniques provide a framework for a cognitive model of writing skills from which these procedures can be compared. It is suggested that practitioners interested in reliably measuring all aspects of the proposed writing process continuum, as characterized by this cognitive model, use both indirect and direct methods.An issue in ability and achievement testing that has gained increased attention in recent psychometric literature is the relative value of information gained from objective (multiple-choice) and freeresponse (including essay) testing methodologies.Critics of objective testing claim that the information that can be gained from multiple-choice tests is limited relative to that gained from freeresponse options. Others claim that the scoring reliability and validity of essay and other free-response formats is so poor as to outweigh any such advantage.Research on the comparability of the construct validity of the two general classes of measures has been somewhat sparse and varied in terms of conclusions. Early work comparing multiple-choice tests with constructed-response tests (Davis & Fifer, 1959;Heim & Wats, 1967; Vemon, 1962) generally indicated that tests employing different formats cannot be expected to have the same means, standard deviations, and correlations with criterion variables. Some of these differences can be assumed to be due to changes in the scale of measurement and amount of error variance associated with each format. Thus the results of this earlier work do not necessarily imply lack of construct equivalence. Traub and Fisher (1977) recognized these problems and employed methodology that equated scale parameters and error variances on three response formats for both verbal and quantitative measures. Two of these formats were multiple-choice and constructed-response. Using confirmatory factor analysis ~c~',~)9 Traub and Fisher found little evidence of a format effect for the mathematical reasoning items, and only weak evidence that the freeresponse and items were measuring different constructs for verbal comprehension items.Ward, Frederiksen, and Carlson ( 19~0) also using CFA, compared machine-scored and constructed-response forms of a test of ability to formulate scientific hypotheses. While their data were somewhat restricted and their analysis was more concerned with correlations of the resulting scores with personality and other cognitive variables, the Ward et al. results indicated that the two formats measure, at least in part, different constructs. In an additional study, Ward (1982) concluded that at Purdue University on June 18, 2015 apm.sagepub.com Downloaded from 118 for verbal aptitude items, various item formats produce much the same information and are essentially equivalent in terms of both the technical adequacy of the resulting measures and the construct interpretations of the resulting scores.With the except...

show abstract

AN EXPERIMENT ON MULTIPLE‐CHOICE VERSUS OPEN‐ENDED ANSWERING IN A VOCABULARY TEST

Cited by 24 publications

References 0 publications

Comparison of Difficulties and Reliabilities of Quantitative Word Problems in Completion and Multiple-Choice Item Formats

Comparison of Difficulties and Reliabilities of Quantitative Word Problems in Completion and Multiple-Choice Item Formats

Test Item Construction and Validation: Developing a Statewide Assessment for Agricultural Science Education

A Comparison of the Information Provided by Essay, Multiple-Choice, and Free-Response Writing Tests

Contact Info

Product

Resources

About