Appraising item equivalence across multiple languages and cultures

Abstract

Activity in the area of language testing is expanding beyond second language acquisition. In many contexts, tests that measure language skills are being translated into several different languages so that parallel versions exist for use in multilingual contexts. To ensure that translated items are equivalent to their original versions, both statistical and qualitative analyses are necessary. In this article, we describe a statistical method for evaluating the translation equivalence of test items that are scored dichotomously. We provide an illustration of the method to a portion of the verbal subtest of the Psychometric Entrance Test, which is a large-scale postsecondary admissions test used in Israel. By evaluating translated items statistically, language test-developers can ensure the comparability of tests across languages and they can identify the types of problems that should be avoided in future translation efforts.

Equivalence of Translations in International Reading Literacy Studies

“…These syntactic differences and problems, however, are normally less important than semantic differences and problems (Binkley & Pignal, 1998;Kirsch & Mosenthal, 1990;Sireci & Allalouf, 2003). Semantically the most serious problems are those caused by words for which it is difficult to find a common definition across languages, by metaphors and idioms, and by abstract and generic terms (Bechger et al, 1998;Manesse, 2000;OECD, 2004b).…”

“…Often, the texts used in these studies have mainly represented Western and Anglo-Saxon cultures (see also Hamilton & Barton, 2000), with their contents and conventions consequently being less relevant and familiar in non-Western and non-AngloSaxon cultures (see also Sireci & Allalouf, 2003). Also, attempts to minimize this bias by doing away with texts or items showing cross-cultural differences have frequently been unsuccessful, because they have commonly resulted in the texts being culturally neutral or representing a US-based transnational culture (Hamilton & Barton, 2000).…”

“…Semantically the most serious problems are those caused by words for which it is difficult to find a common definition across languages, by metaphors and idioms, and by abstract and generic terms (Bechger et al, 1998;Manesse, 2000;OECD, 2004b). Problems also arise when the associative meanings or, for example, the stylistic connotations of a word cannot be carried over to the target language (Arffman, 2002); when a choice has to be made between a common and scientific term (OECD, 2004b); and when words are taken out of context (Elley, 1998;Sireci & Allalouf, 2003). Semantic problems often result in the words in the source and target texts not being equally familiar and in the texts differing in style and in, for example, formality (Arffman, 2002).…”

“…non-Indo-European) and cultures (e.g. Middle-East and Asian) (Sireci & Allalouf, 2003; see also . In the French study, moreover, it is claimed that the French translations were of poor quality and contained less precise terms than the English originals, as well as lexical and pronominal errors; it has been maintained that there were considerable differences between the French and English texts, and equivalence of difficulty was not attained (Manesse, 2000).…”

deltaPlotR: AnRPackage for Differential Item Functioning Analysis with Angoff's Delta Plot

“…Indeed, the delta plot does not rely on sophisticated statistics with known asymptotic distribution, such as e.g., the Mantel-Haenszel method or logistic regression. Its practical usefulness and interest for practitioners can be assessed by recent DIF studies using the delta plot (Abedalaziz 2010;Facon, Magis, and Courbois 2012a;Facon and Nuchadee 2010;Facon, Nuchadee, and Bollengier 2012b;Michaelides 2010;Moon, McLean, and Kaufman 2003;Robin, Sireci, and Hambleton 2003;Sireci and Allalouf 2003;Sireci, Patsula, and Hambleton 2005;Van Herwegen, Farran, and Annaz 2011).…”