Validity Issues in Assessing Linguistic Minorities

Abstract

Throughout the world, tests are administered to some examinees who are not fully proficient in the language in which they are being tested. It has long been acknowledged that proficiency in the language in which a test is administered often affects examinees' performance on a test. Depending on the context and intended uses for a particular assessment, linguistic proficiency may be relevant to the tested construct and subsequent interpretations, or may be a source of construct-irrelevant variance that undermines accurate interpretation of the test performance of linguistic minorities who are not proficient in the language of the assessment. In this article, we highlight key validity issues to be considered when testing linguistic minorities, regardless of whether language is central or construct-irrelevant. We discuss examples of the different types of studies test users and developers could conduct to evaluate the validity of scores of linguistic minorities. These issues span test development and validation activities. We conclude with a list of critical factors to consider in test development and evaluation whenever linguistic minorities are tested.

Rethinking “(Under) Performance” for Black English Speakers: Beyond Achievement to Opportunity

“…Notwithstanding, this finding raises questions about the ways in which language mediates literacy achievement for Black subpopulations of youth. As has been previously documented, students’ home languages and dialects play a key role in predicting their academic success and, specifically, their literacy learning and assessment results (e.g., Abedi, 2017; Bauer et al, 2017; Faulkner-Bond & Sireci, 2015; Mislevy & Duran, 2014; Siegel, 2012; Skutnabb-Kangas, 2013). And evidence has consistently pointed to the role of AAL as a key predictor in the literacy outcomes of Black American youth (e.g., Craig et al, 2009).…”

A Systematic Examination of the Linguistic Demand of Cognitive Test Directions Administered to School-Age Populations

“…They noted, however, that simplified language would not be a necessary accommodation, if standard item writing guidelines were followed, which include limiting verbosity and reducing overall language load for the examinee. Similarly, Faulkner-Bond and Sireci (2015) noted that, during test development, a sensitivity review should flag items that are particularly linguistically demanding. The methodology and results presented herein could be used to inform the development of cognitive test batteries.…”

Professional Development at Scale: The Causal Effect of Obtaining an SEI Endorsement Under Massachusetts’s RETELL Initiative

“…Our primary analyses evaluate the effect of assignment to an SEI-endorsed teacher on a student’s standardized math and ELA scores, with special consideration for the effect for ELs. A notable limitation of focusing on test scores in our context is that their lack of proficiency in English can interfere with ELs ability to demonstrate their content knowledge on standardized tests (Abedi et al, 2004; Faulkner-Bond & Sireci, 2015; Kieffer et al, 2009; Lane & Leventhal, 2015; Martiniello, 2009; Robinson, 2010). AS our analyses compare ELs with other ELs, this issue is unlikely to bias the estimates, but it could reduce the precision of estimates within models evaluating the impact of the training for ELs.…”