Dual‐scale BERT using multi‐trait representations for holistic and trait‐specific essay grading

Abstract

As automated essay scoring (AES) has progressed from handcrafted techniques to deep learning, holistic scoring capabilities have merged. However, specific trait assessment remains a challenge because of the limited depth of earlier methods in modeling dual assessments for holistic and multi‐trait tasks. To overcome this challenge, we explore providing comprehensive feedback while modeling the interconnections between holistic and trait representations. We introduce the DualBERT‐Trans‐CNN model, which combines transformer‐based representations with a novel dual‐scale bidirectional encoder representations from transformers (BERT) encoding approach at the document‐level. By explicitly leveraging multi‐trait representations in a multi‐task learning (MTL) framework, our DualBERT‐Trans‐CNN emphasizes the interrelation between holistic and trait‐based score predictions, aiming for improved accuracy. For validation, we conducted extensive tests on the ASAP++ and TOEFL11 datasets. Against models of the same MTL setting, ours showed a 2.0% increase in its holistic score. Additionally, compared with single‐task learning (STL) models, ours demonstrated a 3.6% enhancement in average multi‐trait performance on the ASAP++ dataset.

Special issue on speech and language AI technologies

“…However, assessing specific traits remains challenging because of the limited depth of existing methods to model dual assessments for holistic and multitrait tasks. To address this challenge, a paper in this special issue titled "Dual-Scale BERT using Multi-Trait Representations for Holistic and Trait-Specific Essay Grading" [3] by Cho and others explores comprehensive feedback while modeling the interconnections between holistic and trait representations. The authors introduce the DualBERT-Trans-CNN model, which combines transformer-based representations with a novel dualscale BERT encoder at the document level.…”