Identifying Content and Cognitive Skills that Produce Gender Differences in Mathematics: A Demonstration of the Multidimensionality‐Based DIF Analysis Paradigm

Abstract

Progress has been made in developing statistical methods for identifying DIF items, but procedures to aid with the substantive interpretations of these items have lagged behind. To overcome this problem, Roussos and Stout (1996) proposed a multidimensionality‐based DIF analysis paradigm. We illustrate and evaluate an application of this framework as it applied to the study of gender differences in mathematics. Four characteristics distinguish this study from previous research: the substantive analysis was guided by past research on the content and cognitive‐related sources of gender differences in mathematics achievement, as presented in the taxonomy by Gallagher, De Lisi, Holst, McGillicuddy‐De Lisi, Morely, and Cahalan (2000); the substantive analysis was conducted by reviewers who were highly knowledgeable about the cognitive strategies students use to solve math problems; three statistical methods were used to test hypotheses about gender differences, including SIBTEST, DIMTEST, and multiple linear regression; and the data were from a curriculum‐based achievement test developed with the goal of minimizing obvious, content‐related gender differences. We show that the framework can lead to clearly interpretable results and we highlight both the strengths and weaknesses of applying the Roussos and Stout framework to the study of group differences.

Gender Differences in Large-Scale Math Assessments: PISA Trend 2000 and 2003

“…Spatial ability has long been documented as a domain that displays large and robust gender differences. This finding resonates with previous research that reports that males show superior spatial ability on standardized tests (Casey et al, 1995;Gierl et al, 2003;Halpern, 1997;Hanna, 1986;Ryan & Chiu, 2001). In the K-12 curricula in the United States, spatial ability is not systematically taught.…”

“…Spatial ability is the domain that has shown the largest and most robust gender differences in favor of males (Casey, Nuttall, Pezaris, & Benbow, 1995;Gierl, Bisanz, Bisanz, & Boughton, 2003;Halpern, 1997). Items assessing spatial abilities have strong ties to traditional geometry but go far beyond it in content and measures.…”

Are Inferential Reading Items More Susceptible to Cultural Bias Than Literal Reading Items?

“…By contrast, a one-tailed test was conducted for the last four hypotheses given that it was explicitly stated which group would be favored/disadvantaged by the bundles after controlling for overall reading ability. The process is consistent with confirmatory DBF research (see Abbott, 2006Abbott, , 2007Banks, 2006;Douglas et al, 1996;Gierl, Bisanz, Bisanz, & Boughton, 2003;Gierl, Bisanz, Bisanz, Boughton, & Khaliq, 2001;Gierl, & Khaliq, 2001;Mendes-Barnett & Ercikan, 2006;Ryan & Chiu, 2001;Ryan & Fan, 1996;Wolf & Leon, 2009). …”

A Review of Recent Developments in Differential Item Functioning

“…This approach has already been illustrated in several studies (Camilli, 1992;Gierl, Bisanz, Bisanz, & Boughton, 2003;Walker & Beretvas, 2001). Even so, there are only a few promising approaches Stout et al, 1997), and they are unlikely to be used operationally.…”