Impact of Selected Background Variables on Students' NAEP Math Performance

Self Cite

193

149

Using existing data from several locations across the U.S., this study examined the impact of students' language background on the outcome of achievement tests. The results of the analyses indicated that students' assessment results might be confounded by their language background variables. English language learners (ELLs) generally perform lower than non-ELL students on reading, science, and math-a strong indication of the impact of English language proficiency on assessment. Moreover, the level of impact of language proficiency on assessment of ELL students is greater in the content areas with higher language demand. For example, analyses showed that ELL and non-ELL students had the greatest performance differences in the language-related subscales of tests in areas such as reading. The gap between the performance of ELL and non-ELL students was smaller in science and virtually nonexistent in the math computation subscale, where language presumably has the least impact on item comprehension.The results of our analyses also indicated that test item responses by ELL students, particularly ELL students at the lower end of the English proficiency spectrum, suffered from low reliability. That is, the language background of students may add another dimension to the assessment outcome that may be a source of measurement error in the assessment for English language learners.Further, the correlation between standardized achievement test scores and external criterion measures was significantly larger for the non-ELL students than for the ELL students. Analyses of the structural relationships between individual items and between items and the total test scores showed a major difference between ELL and non-ELL students. Structural models for ELL students demonstrated lower statistical EDUCATIONAL ASSESSMENT, 8(3),

Section: Validitymentioning

confidence: 99%

Standardized Achievement Tests and English Language Learners: Psychometrics Issues

Abedi¹

2002

Self Cite

193

149

“…Based on this perspective, much work in this area argues that content area tests may be biased against ELLs because the language of these tests unfairly adds an irrelevant hurdle for ELLs, causing construct-irrelevant variance in ELLs' scores (e.g., Abedi, 2007;Abedi et al, 2000Abedi et al, /2005. Empirical arguments for this perspective include studies that link the presence of complex linguistic features in test items to greater relative difficulty of the items for ELLs (e.g., Martiniello, 2009;Wolf & Leon, 2009), and it is this perspective that has motivated the use of linguistically simplified tests for ELLs as a test accommodation (Abedi et al, 2000(Abedi et al, /2005Abedi, Hofstetter, Baker, & Lord, 2001;Abedi, Hofstetter, & Lord, 2004;Abedi, Lord, & Hofstetter, 1998;Lotherington-Woloszyn, 1993;Rivera & Stansfield, 2001). Kieffer et al (2009) also consider the possibility that the performance gap between ELLs and non-ELLs is caused by different levels of linguistic ability, but in contrast to Abedi (2007), they hypothesize that the performance gaps are caused by differences in proficiency with linguistic skills that may be necessary for content area achievement.…”

mentioning

confidence: 98%

Construct Relevant or Irrelevant? The Role of Linguistic Complexity in the Assessment of English Language Learners’ Science Knowledge

Avenia-Tapper

Llosa

2015

This article addresses the issue of language-related construct-irrelevant variance on content area tests from the perspective of systemic functional linguistics. We propose that the construct relevance of language used in content area assessments, and consequent claims of construct-irrelevant variance and bias, should be determined according to the degree of correspondence between language use in the assessment and language use in the educational contexts in which the content is learned and used. This can be accomplished by matching the linguistic features of an assessment and the linguistic features of the domain in which the assessment is measuring achievement. This represents a departure from previous work on the assessment of English language learners' content knowledge that has assumed complex linguistic features are a source of construct irrelevant variance by virtue of their complexity.

“…In the last decade, a few important empirical studies have investigated the relationship between some of these linguistic features and the difficulty (p value) of math word problems for ELLs and non-ELLs in elementary, middle, and high school levels (Abedi et al, 2005;Abedi & Lord, 2001;Abedi et al, 1998;Abedi et al, 1997;Lord, Abedi, & Poosuthasee, 2000;Shaftel, Belton-Kocher, Glasnapp, & Poggio, 2006). Although many of these studies did, as predicted, find a relationship between linguistic complexity and ELLs' performance in math word problems, the effect of specific linguistic features varied from test to test and from one grade to another.…”

Section: Previous Research On Linguistic Complexity and Math Performamentioning

confidence: 99%

“…According to the literature, the linguistic features of natural language that create comprehension difficulties for ELLs include vocabulary/lexical complexity (number of low-frequency, abstract, ambiguous/polysemous, idiomatic, and culture-specific nonmathematical vocabulary terms) and increased syntactic complexity (measured as mean sentence length in words, item length in words, noun phrase length, number of prepositional phrases, and participial modifiers, presence of syntactically complex sentences; i.e., with relative, complement, adverbial and conditional clauses; Abedi & Lord, 2001;Abedi, Lord, & Hofstetter, 1998;Abedi et al, 1997;Butler, Bailey, Stevens, Huang, & Lord, 2004;Kim-Wolf et al, 2008;Martiniello, 2006aMartiniello, , 2007bMartiniello, , 2008Spanos, Rhodes, Dale, & Crandall, 1988).…”

Section: Previous Research On Linguistic Complexity and Math Performamentioning

confidence: 99%

Linguistic Complexity, Schematic Representations, and Differential Item Functioning for English Language Learners in Math Tests

Martiniello

2009

131

This article examines nonmathematical linguistic complexity as a source of differential item functioning (DIF) in math word problems for English language learners (ELLs). Specifically, this study investigates the relationship between item measures of linguistic complexity, nonlinguistic forms of representation and DIF measures based on item response theory difficulty parameters in a state fourth-grade math test. This study revealed that the greater the item nonmathematical lexical and syntactic complexity, the greater are the differences in difficulty parameter estimates favoring non-ELLs over ELLs. However, the impact of linguistic complexity on DIF is attenuated when items provide nonlinguistic schematic representations that help ELLs make meaning of the text, suggesting that their inclusion could help mitigate the negative effect of increased linguistic complexity in math word problems.