Building a validity argument for the use of academic language tests for immigration purposes: Evidence from immigration-seeking test-takers

Abstract

As validity pertains to test use rather than the test itself, using a test for unintended purposes requires a new validation program using additional evidence from relevant sources. This small-scale study contributes to the validation of the use of originally academic language tests-the International English Language Testing System and the Test of English as a Foreign Language-for assessing skilled immigration eligibility. Data were collected from 39 immigration-seeking test-takers, who are arguably under-represented in validation research. Analysis was informed by contemporary validity theory, which treats validity as a unitary concept incorporating score reliability, score interpretation, score-based decisions and their consequences. Results showed that the test-takers’ perceptions varied widely. The evidence supporting this use included generally positive perceptions of the scores’ reliability, washback effect, and fairness of score-based decisions. The refuting evidence concerned factors perceived to interfere with test-takers’ performance and the complex consequences for the test-takers in aspects other than washback. However, overwhelmingly, as test-takers found the score-based decisions as fair, the validity judgement appeared tilted towards the positive side from the perspectives of these key stakeholders. Although the ultimate validity judgement requires the examination of evidence from other significant stakeholders as well, the present study has contributed valuable and unique evidence and bears important implications for research, practice, and policy particularly in high-stakes contexts such as immigration.

Language Assessment for Immigration: A Review of Validation Research Over the Last Two Decades

“…Only study of Hoang (2019) has touched upon the decisions inference, which investigated 39 test-takers taking IELTS or TOEFL for skilled migration purposes. Participants were invited to finish an online survey followed by an individual interview.…”

“…Six empirical studies are associated with stakeholders’ perceptions of test consequences ( Merrylees, 2003 ; Ahern, 2009 ; Merrifield, 2012 ; Rumsey et al, 2016 ; Frost, 2017 ; Hoang, 2019 ); and the stakeholders mainly comprise immigration-seekers, immigrants, and key personnel in the immigration authorities. Overall, stakeholders’ perceptions are categorized into positive , negative , and compounded .…”