Rise of the machines? The evolving role of AI technologies in high-stakes assessment

Abstract

Our world has been transformed by technologies incorporating artificial intelligence (AI) within mass communication, employment, entertainment and many other aspects of our daily lives. However, within the domain of education, it seems that our ways of working and, particularly, assessing have hardly changed at all. We continue to prize examinations and summative testing as the most reliable way to assess educational achievements, and we continue to rely on paper-based test delivery as our modus operandi. Inertia, tradition and aversion to perceived risk have resulted in a lack of innovation (James, 2006), particularly so in the area of high-stakes assessment. The summer of 2020 brought this deficit into very sharp focus with the A-level debacle in England, where grades were awarded, challenged, rescinded and reset. These events are potentially catastrophic in terms of how we trust national examinations, and the problems arise from using just one way to define academic success and one way to operationalize that approach to assessment. While sophisticated digital learning platforms, multimedia technologies and wireless communication are transforming what, when and how learning can take place, transformation in national and international assessment thinking and practice trails behind. In this article, we present some of the current research and advances in AI and how these can be applied to the context of high-stakes assessment. Our discussion focuses not on the question of whether we should be using technologies, but on how we can use them effectively to better support practice. An example from one testing agency in England using a globally popular test of English that assesses oral, aural, reading and written skills is described to explain and propose just how well new technologies can augment assessment theory and practice.

Technology-enabled Learning in Digital Education: Research Areas and Prediction Trends of Learning Technologies for Authentic Assessments Practices

“…Papers in cluster include the study of the use of AI for grading of authentic assessments. Richardson and Clesham (2021) discussed the efficacy of automated grading systems to facilitate authentic assessment, and shared a case study of automated grading using the Intelligent Essay Assessor scoring engine.…”

AI-Assisted Enhancement of Student Presentation Skills: Challenges and Opportunities

“…Evaluation is challenging; naturally, AI will not be appropriate for assessment in some parts of the curriculum [38]. Furthermore, ensuring the reliability of evaluating speaking and writing skills (that is, open-ended, productive skills) has always been a significant challenge [39,40].…”

High-stakes online assessments: A case study of National Benchmark Tests during COVID-19

“…The coronavirus (COVID-19) pandemic has changed the educational landscape for most students, teachers and lecturers in 2020. The digital transformation that schools, colleges and universities have been slow to adopt, particularly in assessment modes and practices (Richardson & Clesham, 2021), have been accelerated dramatically during the pandemic, with the higher education sector seeking solutions to teaching, learning and assessment. Accelerated digital adoption across all sectors, including education, and higher education in particular, necessitated quick responses to the question of the mode of assessment.…”

“…Comprehensive global reviews were collated by UNESCO (2020a) in the working document summarising national coping strategies on high-stakes examinations and assessments in the education sector. However, what is evident is that in the context of a global pandemic, the widespread reliance on paper-based administration for high-stakes examinations is problematic and we need to continue to seek alternative ways of test delivery (Richardson & Clesham, 2021).…”