Exploring the Impact of Handwriting Recognition on the Automated Scoring of Handwritten Student Answers

Gold, Christian; Zesch, Torsten

doi:10.1109/icfhr2020.2020.00054

Cited by 5 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main challenge with such a procedure is how to deal with the additional errors that might be introduced by the transcription step. The current performance of handwriting recognition is sufficient for scoring (Gold & Zesch, 2020), especially when combined with a personalized handwriting recognition model (Gold et al., 2021). The remaining recognition mistakes are rather on the level of spelling mistakes (for which we already have discussed that they do not influence performance that much).…”

Section: Performance Of Automatic Content Scoringmentioning

confidence: 99%

To Score or Not to Score: Factors Influencing Performance and Feasibility of Automatic Content Scoring of Text Responses

Zesch

Horbach

Zehner

2023

Educational Measurement

Self Cite

View full text Add to dashboard Cite

In this article, we systematize the factors influencing performance and feasibility of automatic content scoring methods for short text responses. We argue that performance (i.e., how well an automatic system agrees with human judgments) mainly depends on the linguistic variance seen in the responses and that this variance is indirectly influenced by other factors such as target population or input modality. Extending previous work, we distinguish conceptual, realization, and nonconformity variance, which are differentially impacted by the various factors. While conceptual variance relates to different concepts embedded in the text responses, realization variance refers to their diverse manifestation through natural language. Nonconformity variance is added by aberrant response behavior. Furthermore, besides its performance, the feasibility of using an automatic scoring system depends on external factors, such as ethical or computational constraints, which influence whether a system with a given performance is accepted by stakeholders. Our work provides (i) a framework for assessment practitioners to decide a priori whether automatic content scoring can be successfully applied in a given setup as well as (ii) new empirical findings and the integration of empirical findings from the literature on factors that influence automatic systems' performance.

show abstract

Section: Performance Of Automatic Content Scoringmentioning

confidence: 99%