The Relationship Between Test Item Format and Gender Achievement Gaps on Math and ELA Tests in Fourth and Eighth Grades

Abstract

Prior research suggests that males outperform females, on average, on multiple-choice items compared to their relative performance on constructed-response items. This paper characterizes the extent to which gender achievement gaps on state accountability tests across the United States are associated with those tests’ item formats. Using roughly 8 million fourth- and eighth-grade students’ scores on state assessments, we estimate state- and district-level math and reading male-female achievement gaps. We find that the estimated gaps are strongly associated with the proportions of the test scores based on multiple-choice and constructed-response questions on state accountability tests, even when controlling for gender achievement gaps as measured by the National Assessment of Educational Progress (NAEP) or Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) assessments, which have the same item format across states. We find that test item format explains approximately 25% of the variation in gender achievement gaps among states.

Effects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response

“…As men often outperformed women in MC versions of performance tests while test takers of both sexes performed approximately equal in FR versions according to recent studies (e.g., DeMars, 1998;Stanger-Hall, 2012;Reardon et al, 2018), we hypothesized an interaction between sex and response format. According to our data, male pupils outperformed female pupils in both response formats, whereas sex and response format did not interact significantly.…”

“…Regarding research on specific cognitive abilities or factual knowledge, data showed that men often outperformed women in the MC version, while both sexes performed approximately the same in the FR version, or women performed even better: these interaction effects were demonstrated, for example, in mathematics and languages (Bolger and Kellaghan, 1990;Reardon et al, 2018), history (Bridgeman and Lewis, 1994) as well as in natural sciences (DeMars, 1998;Stanger-Hall, 2012). In contrast, Bonner (2013) reported no interaction between sex and response format in crystallized intelligence.…”

Language in science performance: do good readers perform better?

“…To avoid confounding effects, we only included items with closed-response formats. Open-response formats were excluded due to the additionally required productive language skills (Brown and Hudson 1998) and due to the effects of response formats on performance (DeMars 2000;Härtig et al 2015;Reardon et al 2018). Furthermore, items with pictures were excluded, because it is known that processing texts with pictures may have favorable effects on reading comprehension (Carney and Levin 2002;Mayer 1989Mayer , 2001Mayer and Gallini 1990).…”

Systematic Review: Research Trends in Underserved Students

“…They have to be aware of the needs and priorities of underserved students to ensure no citizens are left behind in their country, especially in education fields. It's a responsibility of every policy makers to practice 'education is for all' perception in their respective countries (Gershenson, 2013;Ruth et al, 2016;Borman, Grigg, & Hanselman, 2016;Paul et al, 2016;Quinn & Cooc, 2015;Rachel, 2017;Adelson, Dickinson, & Cunningham, 2016;Holt & Duffy, 2017;Valant & Newark, 2016;Reardon, et al, 2018;David, North, & Celia 2016;Drake, 2014). The issues faced by underserved students are not something new.…”

“…There is no equal treatment between female students and male students in academics (Adelson, Dickinson, & Cunningham, 2016). Gender norms and stereotypes are very common in academics (Reardon, Kalogrides, Fahle, Podolsky, & Zarate, 2018). Gender inequality occurs when female students are not being served the same as the male students especially in STEM field.…”