Comparison of a randomized multiple choice format with a written one‐hour physics problem test

Harke, Douglas J.; Herron, J. Dudley; Lefler, R. W.

doi:10.1002/sce.3730560416

Cited by 4 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, there is evidence for construct-equivalence between MC and CR formats in mathematical reasoning items ( Traub and Fisher 1977 ). Further studies of numerical problem solving also revealed very high correlations ( r = .90) between response formats ( Harke et al 1972 ; Horn 1966 ). Regarding knowledge questions, it was also shown that CR formats add little information beyond MC-format items ( Lukhele et al 1994 ; Thissen et al 1994 ).…”

Section: Introductionmentioning

confidence: 94%

Stop Worrying about Multiple-Choice: Fact Knowledge Does Not Change with Response Format

et al. 2022

View full text Add to dashboard Cite

Declarative fact knowledge is a key component of crystallized intelligence. It is typically measured with multiple-choice (MC) items. Other response formats, such as open-ended formats are less frequently used, although these formats might be superior for measuring crystallized intelligence. Whereas MC formats presumably only require recognizing the correct response to a question, open-ended formats supposedly require cognitive processes such as searching for, retrieving, and actively deciding on a response from long-term memory. If the methods of inquiry alter the cognitive processes involved, mean-changes between methods for assessing declarative knowledge should come along with changes in the covariance structure. We tested these assumptions in two online studies administering declarative knowledge items in different response formats (MC, open-ended, and open-ended with cues). Item difficulty clearly increases in the open-ended methods although effects in logistic regression models vary slightly across items. Importantly, latent variable analyses suggest that the method of inquiry does not affect what is measured with different response formats. These findings clearly endorse the position that crystallized intelligence does not change as a function of the response format.

show abstract

Section: Introductionmentioning

confidence: 94%

Stop Worrying about Multiple-Choice: Fact Knowledge Does Not Change with Response Format

et al. 2022

View full text Add to dashboard Cite

show abstract

“…but 'What is true about studies like these that could have been done?' " (Hedges, 1994, This perspective is important particularly in this area of research that hopes to inform test design by providing information about what might likely result from the use of MC and CR items to measure similar or dissimilar constructs. The random effects model gives us the additional room to infer to specific tests whose characteristics have not been explicitly studied to date.…”

Section: Statistical Modeling Considerationsmentioning

confidence: 99%

Construct Equivalence of Multiple‐Choice and Constructed‐Response Items: A Random Effects Synthesis of Correlations

Rodriguez

2003

J Educational Measurement

111

112

View full text Add to dashboard Cite

A thorough search of the literature was conducted to locate empirical studies investigating the trait or construct equivalence of multiple-choice (MC) and constructed-response (CR) items. O f the 67 studies identified, 29 studies included 56 correlations between item in both f o m t s . These 56 correlations were corrected for attenuation and synthesized to establish evidence for a common estimate of correlation (true-score correlations). The 56 disattenuated correlations were highly heterogeneous. A search for moderators to explain this variation uncovered the role of the design characteristics of test items used in the studies. When items are constructed in both formats using the same stem (stem equivalent), the mean corretation between the two formats approaches unity and is signijkantly higher than when using non-stem-equivalent items (particularly when using essay-type items). Construct equivalence, in part, appears to be a function of the item design method or the item writer's intent.

show abstract

“…In addition, there is evidence for construct-equivalence between MC and CR formats in mathematical reasoning items (Traub & Fisher, 1977). Further studies of numerical problem solving also revealed very high correlations (r = .90) between response formats (Harke et al, 1972;Horn, 1966).…”

Section: Format Items Vs Cr Format Itemsmentioning

confidence: 92%

Stop Worrying About Multiple-Choice: Fact Knowledge Does Not Change With Response Format

Goecke¹,

Staab²,

Schittenhelm³

et al. 2022

Preprint

View full text Add to dashboard Cite

Declarative fact knowledge is a key component of crystallized intelligence. It is typically measured with multiple-choice (MC) items. Other response formats, such as open-ended formats are less frequently used, although these formats might be superior for measuring crystallized intelligence. Whereas MC-formats presumably only require recognizing the correct response to a question, open-ended formats supposedly require cognitive processes such as searching for, retrieving, and actively deciding for a response from long-term memory. If the methods of inquiry alter the cognitive processes involved, mean-changes between methods for assessing declarative knowledge should come along with changes in the covariance structure. We tested these assumptions in two online studies administering declarative knowledge items in different response formats (MC, open-ended, and open-ended with cues). Item difficulty clearly increases in the open-ended methods although effects in logistic regression models vary slightly across items. Importantly, latent variable analyses suggest that the method of inquiry does not affect what is measured with different response formats. These findings clearly endorse the position that crystallized intelligence does not change as a function of the response format.

show abstract

Comparison of a randomized multiple choice format with a written one‐hour physics problem test

Cited by 4 publications

References 1 publication

Stop Worrying about Multiple-Choice: Fact Knowledge Does Not Change with Response Format

Stop Worrying about Multiple-Choice: Fact Knowledge Does Not Change with Response Format

Construct Equivalence of Multiple‐Choice and Constructed‐Response Items: A Random Effects Synthesis of Correlations

Stop Worrying About Multiple-Choice: Fact Knowledge Does Not Change With Response Format

Contact Info

Product

Resources

About