Using Algebra Word Problems to Assess Quantitative Ability: Attributes, Strategies, and Errors

Sebrechts, Marc M.; Enright, Mary K.; Bennett, Randy Elliot; Martin, Kathleen J.

doi:10.1207/s1532690xci1403_2

Cited by 29 publications

(28 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Randy Bennett took the lead at ETS in exploring the technology for scoring constructed responses in concert with theory about the relevant constructs, including mathematics Bennett et al 1999Bennett et al , 2000aSandene et al 2005;Sebrechts et al 1991Sebrechts et al , 1996, computer science (Bennett and Wadkins 1995), graphical items (Bennett et al 2000a;, and formulating hypotheses (Bennett and Rock 1995). The scoring of mathematics items has reached a significant level of maturity (Fife 2013), as has the integration of task design and automated scoring (Graf and Fife 2012).…”

Section: Automated Scoringmentioning

confidence: 99%

“…He evaluated the SAT grid-in format with GRE items and concluded that the multiple-choice and grid-in versions of GRE items behaved very similarly. Following the adoption of the grid-in format in the SAT, a more comprehensive examination of mathematics item formats that could serve to elicit quantitative skills was undertaken, informed by advances in the understanding of mathematical cognition and a maturing computerbased infrastructure (Bennett and Sebrechts 1997;Bennett et al , 1999Bennett et al , 2000aSandene et al 2005;Sebrechts et al 1996). More recently, the mathematics strand of the CBAL initiative has attempted to unpack mathematical proficiency by means of competency models, the corresponding constructed-response tasks (Graf 2009), and scoring approaches (Fife 2013).…”

Section: Mathematicsmentioning

confidence: 99%

See 1 more Smart Citation

A Historical Survey of Research Regarding Constructed-Response Formats

Bejar

2017

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

This chapter chronicles ETS research and development contributions related to the use of constructed-response item formats. 1 The use of constructed responses in testing dates back to imperial China, where tests were used in the selection of civil servants. However, in the United States, the multiple-choice format became dominant during the twentieth century, following its invention and use by the SAT ® examinations created by the College Board in 1926. When ETS was created in 1947, post-secondary admissions testing was largely based on tests consisting of multiplechoice items. However, from the start, there were two camps at ETS: those who believed that multiple-choice tests were sufficiently adequate for the purpose of assessing "verbal" skills and those who believed that "direct" forms of assessment requiring written responses had a role to play. For constructed-response formats to regain a foothold in American education several hurdles would need to be overcome. Research at ETS was instrumental in overcoming those hurdles.The first hurdle was that of reliability, specifically the perennial issue of low interrater agreement, which plagued the acceptance of constructed-response formats for most of the twentieth century. The second hurdle was broadening the conception of validity to encompass more than predictive considerations, a process that began with the introduction of construct validity by Cronbach and Meehl (1955). Samuel Messick at ETS played a crucial role in this process by making construct validity relevant to educational tests. An inflexion point in the process of reincorporating constructed-response formats more widely in educational tests was marked 1 Constructed responses to a prompt or question can range in scope and complexity. Perhaps the most common constructed response is the written essay. However, short written responses to questions are also considered to be constructed, as are spoken answers in response to a prompt, mathematical responses (equations, plotted functions, etc.), computer programs, and graphical responses such as architectural designs.

show abstract

Section: Automated Scoringmentioning

confidence: 99%

Section: Mathematicsmentioning

confidence: 99%

A Historical Survey of Research Regarding Constructed-Response Formats

Bejar

2017

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

show abstract

“…For complex problems that involve more that one triad, problem structure describes the way that these triads are linked. Shalin and Bee found that many two-step arithmetic word problems could be classified as exemplars of one of a number of structures (such as hierarchy, shared-whole, and shared-part), and that these problem structures had an effect on problem difficulty This idea can be extended to other word problems, and the kind of superordinate constraint that allows the subparts of a problem to be composed can be used as one feature in classifying problems (Hall et aI., 1989;Sebrechts, Enright, Bennett, & Martin, 1996). For example, round trip problems (the distance on one part of the trip equals the distance on the second part of the trip) exemplify a class of problems in which the superordinate constraint can be described as Distance I = Distance 2.…”

Section: Researchon Word Problemsmentioning

confidence: 99%

Items by Design: The Impact of Systematic Feature Variation on Item Statistical Characteristics

Enright

Morley

Sheehan

1999

ETS Research Report Series

View full text Add to dashboard Cite

We wish to recognize the contribution of the many test development staff members whose advice and cooperationwas essential to this project. Specialthanks to Jackie Tchorni, Judy Smith, and Jutta Levin. We also appreciateBob Mislevy's advice about how to estimate the usefulness of collateralinformation. Finally, we are grateful to the Graduate RecordExaminations Board for supportingthis research. AbstractThis study investigated the impact of systematic item feature variation on item statistical characteristics and the degree to which such information could be used as collateral information to supplement examinee performance data and reduce pretest sample size. Two families of word problem variants for the quantitative section of the Graduate Record Examination (GRE®) General Test were generated by systematically manipulating item features. For rate problems, the item design features affected item difficulty (Adj. R 2 = .90), item discrimination (Adj. R 2 = .50), and guessing (Adj. R 2 = .41). For probability problems the item design features affected difficulty (Adj. R 2 = .61), but not discrimination orguessing. The results demonstrate the enormous potential of systematically creating item variants. However, questions of how best to manage variants in item pools and to implement statistical procedures that use collateral information must still be resolved.

show abstract

“…문장제 풀이는 충분한 개념적 지식과 수들의 부분-부분-전체 관계에 대한 지식, 통합되고 잘 조직된 지식 기반과 메타인지 과정에 의존한다 (Ng & Lee, 2009 Sebrechts, Enright, Bennet와 Martin(1996)는 수학 적 문제 해결 과정을 크게 네 가지의 인지 활동인 "문제 번역(Problem Translation)", "문제 통합(Problem Integration)", "풀이 계획 및 모니터링(Solution Planning and Monitoring)", "풀이 실행(Solution Execution)"으로 제시하였다. Sebrechts et al(1996) …”

unclassified

An Analysis of the Relationship between Students' Understanding and their Word Problem Solving Strategies of Multiplication and Division of Fractions

Kim¹,

Whang²

2011

The Mathematical Education

View full text Add to dashboard Cite

Using Algebra Word Problems to Assess Quantitative Ability: Attributes, Strategies, and Errors

Cited by 29 publications

References 23 publications

A Historical Survey of Research Regarding Constructed-Response Formats

A Historical Survey of Research Regarding Constructed-Response Formats

Items by Design: The Impact of Systematic Feature Variation on Item Statistical Characteristics

An Analysis of the Relationship between Students' Understanding and their Word Problem Solving Strategies of Multiplication and Division of Fractions

Contact Info

Product

Resources

About