Developer involvement and COI disclosure in high-stakes English proficiency test validation research: A systematic review

Open Access in Language Testing and Assessment: The Case of Two Flagship Journals

“…Commercialisation of research via university-industry collaboration, which is not uncommon in language testing research, may prohibit the sharing of proprietary materials and (operational) test data (see Isbell & Kim, 2023 for a recent review of developer involvement in language testing). For instance, a research team at a university may receive funding from a test developer (e.g., Cambridge Assessment English, Duolingo, Educational Testing Service, Pearson) to conduct research on a large-scale standardized exam and the proprietary materials and data would typically be kept confidential.…”

“…Language testing, as a research field with substantial commercial and industry presence, is faced with unique challenges relating to open access, relative to other fields in applied linguistics (Burton, 2023;Isbell & Kim, 2023). On the one hand, commercial interests, legal considerations, and test security concerns influence and sometimes prohibit data and materials sharing; on the other hand, public transparency and accountability are particularly needed in language testing, especially when it comes to high-stakes tests and related validation research.…”

“…Conversely, articles focusing on commercial tests, or a mixture of commercial and non-commercial tests were found to be less likely to have open materials. This could be a reflection of the unique challenges to open science faced by the field of language testing (Burton, 2023;Isbell & Kim, 2023). This tension between the free flow of scientific knowledge and the commercialisation of scientific discoveries (De Silva & Vance, 2017), particularly in relation to test security (Section 2.2.3), is a challenge that the field needs to grapple with.…”

Purposeful turns for more equitable and transparent publishing in language testing and assessment

“…Second, there has recently been a spotlight on conflict-of-interest declarations in our field as part of wider conversations on research ethics in applied linguistics. In Isbell and Kim's (2023) systematic review on conflict-of-interest reporting in high-stakes language proficiency testing research, a third of the included articles were published in Language Testing-the highest proportion of the five journals they examined in their search. The authors revealed that just 4% to 7% of included articles made appropriate conflict-ofinterest disclosures, highlighting the need for substantial improvement.…”

“…We also require conflict-of-interest declarations from authors and outline our requirements, drawing on guidance from the Committee on Publication Ethics (2023) and precedents from other fields (e.g., International Committee of Medical Journal Editors [ICJME], 2023). We believe that these efforts, which implement Isbell and Kim's (2023) recommendations, are long overdue and necessary to pave the way forward. We also understand that an overhaul of the International Language Testing Association's (ILTA) Code of Ethics will be conducted in the coming months and that managing and disclosing conflicts of interests is likely to be an area of focus (Deygers & Malone, 2023).…”