Strengths and weaknesses of automated scoring of free-text student answers

Abstract

Free-text tasks, where students need to write a short answer to a specific question, serve as a well-established method for assessing learner knowledge. To address the high cost of manually scoring these tasks, automated scoring models can be used. Such models come in various types, each with its own strengths and weaknesses. Comparing these models helps in selecting the most suitable one for a given problem. Depending on the assessment context, this decision can be driven by ethical or legal considerations. When implemented successfully, a scoring model has the potential to substantially reduce costs and enhance the reliability of the scoring process. This article compares the different categories of scoring models across a set of crucial criteria that have immediate relevance to model employment in practice.