Targeted Linguistic Simplification of Science Test Items for English Learners

Abstract

In this experimental study, 20 multiple-choice test items from the Massachusetts Grade 5 science test were linguistically simplified, and original and simplified test items were administered to 310 English learners (ELs) and 1,580 non-ELs in four Massachusetts school districts. This study tested the hypothesis that specific linguistic features of test items contributed to construct-irrelevant variance in science test scores of ELs. Simplifications targeted specific linguistic features, to identify those features with the largest impacts on ELs’ test performance. Of all the linguistic simplifications used in this study, adding visual representations to answer choices had the largest positive effect on ELs’ performance. These findings have significant implications for the design of multiple-choice test items that are fair and valid for ELs.

An investigation of item, examinee, and country correlates of rapid guessing in PISA

“…Although this recommendation follows best practice guidelines (Haladyna & Downing, 1989), recent reviews of operational testing programs suggest that current items often include superfluous information that does not improve measurement of the intended construct (Haladyna et al, 2019). Reducing linguistic complexity may not only benefit the reduction of RG, but may also advance the validity of score-based inferences for examinees tested in their non-native language (e.g., Noble et al, 2020) -a group found to engage in RG at higher rates than their native language counterparts.…”

Do Students With Specific Learning Disorders With Impairments in Reading Benefit From Linguistic Simplification of Test Items in Science?

“…This effect refers to the phenomenon that more information can be acquired when learning information is presented both verbally and visually because these two types of information are processed independently. Studies indicate that the combination of both LS and the multimedia effect positively affects science performance for a general student population (e.g., Siegel, 2007), nonnative students (e.g., Noble et al, 2020), and students with learning disorders (Kettler et al, 2012). Hence, combining LS and the multimedia effect to reduce linguistic complexity in support of students with SLD-IR might be a promising project to pursue in future research.…”

Increasing the Utility of the Comprehensive Assessment of Psychopathic Personality–Lexical Rating Scale (CAPP-LRS): Instrument Adaptation and Simplification

“…Examining literature drawn from the area of standardized academic achievement testing among English learners provides some insight into best practices for the modification of assessment instruments to reduce construct irrelevant linguistic complexity. For example, a large body of research has examined the impact of minor wording changes in test items to clarify the construct under assessment (e.g., scientific/mathematical knowledge) in academic settings among students with lower verbal skills, with results demonstrating that syntactic and vocabulary complexity has a significant impact on performance ( Abedi et al, 2005 ; Abedi & Lord, 2001 ; Noble, et al, 2020 ; Rivera & Stansfield, 2001 ). An important feature of successful linguistic accommodation concerns maintaining the integrity and validity of the assessment; that is, such modifications should not compromise the construct under study.…”