A Trait-based Deep Learning Automated Essay Scoring System with Adaptive Feedback

Hussein, Mohamed A.; Hesham, A; Nassef, Mohammad

doi:10.14569/ijacsa.2020.0110538

Cited by 13 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To evaluate the performance of our model, we use the Quadratic Weighted Kappa (QWK) metric, which has been widely-adopted in both holistic essay scoring and essay trait scoring research (Alikaniotis, Yannakoudakis, and Rei 2016;Cummins, Zhang, and Briscoe 2016;Phandi, Chai, and Ng 2015;Chen and He 2013;Dong and Zhang 2016;Dong, Zhang, and Yang 2017;Jin et al 2018;Ridley et al 2020;Mathias and Bhattacharyya 2020;Hussein, Hassan, and Nassef 2020) and is designed to measure the level of agreement between two raters.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Automated Cross-prompt Scoring of Essay Traits

Ridley

Dai

et al. 2021

AAAI

View full text Add to dashboard Cite

The majority of current research in Automated Essay Scoring (AES) focuses on prompt-specific scoring of either the overall quality of an essay or the quality with regards to certain traits. In real-world applications obtaining labelled data for a target essay prompt is often expensive or unfeasible, requiring the AES system to be able to perform well when predicting scores for essays from unseen prompts. As a result, some recent research has been dedicated to cross-prompt AES. However, this line of research has thus far only been concerned with holistic, overall scoring, with no exploration into the scoring of different traits. As users of AES systems often require feedback with regards to different aspects of their writing, trait scoring is a necessary component of an effective AES system. Therefore, to address this need, we introduce a new task named Automated Cross-prompt Scoring of Essay Traits, which requires the model to be trained solely on non-target-prompt essays and to predict the holistic, overall score as well as scores for a number of specific traits for target-prompt essays. This task challenges the model's ability to generalize in order to score essays from a novel domain as well as its ability to represent the quality of essays from multiple different aspects. In addition, we introduce a new, innovative approach which builds on top of a state-of-the-art method for cross-prompt AES. Our method utilizes a trait-attention mechanism and a multi-task architecture that leverages the relationships between each trait to simultaneously predict the overall score and the score of each individual trait. We conduct extensive experiments on the widely used ASAP and ASAP++ datasets and demonstrate that our approach is able to outperform leading prompt-specific trait scoring and cross-prompt AES methods.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) Bhattacharyya 2018, 2020;Hussein, Hassan, and Nassef 2020).…”

Section: Introductionmentioning

confidence: 99%

Automated Cross-prompt Scoring of Essay Traits

Ridley

Dai

et al. 2021

AAAI

View full text Add to dashboard Cite

show abstract

“…-Integrating multiple AES models (Aomi et al 2021) 2. Prompt-specific trait scoring -Multiple trait-specific models (Mathias et al 2020) -Model with multiple output modules (Hussein et al 2020) 3. Cross-prompt holistic scoring -Convolution layer This layer captures local textual dependencies using convolution neural networks (CNNs) from a sequence of word-embedding vectors.…”

Section: Rnn-based Modelmentioning

confidence: 99%

“…Concretely, in their study, the AES model was trained for each trait independently, and predicted scores using trait-specific models. Hussein et al (2020) proposed a model specialized in a prompt-specific trait scoring task that can predict multiple trait scores jointly. The model is formulated as a multioutput model based on the RNN-based model (Taghipour and Ng 2016), introduced in Sect.…”

Section: Prompt-specific Trait Scoringmentioning

confidence: 99%

A review of deep-neural automated essay scoring models

Uto

2021

Behaviormetrika

View full text Add to dashboard Cite

Automated essay scoring (AES) is the task of automatically assigning scores to essays as an alternative to grading by humans. Although traditional AES models typically rely on manually designed features, deep neural network (DNN)-based AES models that obviate the need for feature engineering have recently attracted increased attention. Various DNN-AES models with different characteristics have been proposed over the past few years. To our knowledge, however, no study has provided a comprehensive review of DNN-AES models while introducing each model in detail. Therefore, this review presents a comprehensive survey of DNN-AES models, describing the main idea and detailed architecture of each model. We classify the AES task into four types and introduce existing DNN-AES models according to this classification.

show abstract

“…Because of this discovery, some researchers have applied neural network models to automatic scoring tasks. For example, Qian Hussein et al [25] built an adaptive deep learning speaking scoring system. Bshary et al [26] used the LSTM to evaluate oral pronunciation.…”

Section: Introductionmentioning

confidence: 99%

An automatic scoring method for Chinese-English spoken translation based on attention LSTM

Guo¹

2018

ICST Transactions on Scalable Information Systems

View full text Add to dashboard Cite

In this paper, we propose an automatic scoring method for Chinese-English spoken translation based on attention LSTM. We select semantic keywords, sentence drift and spoken fluency as the main parameters of scoring. In order to improve the accuracy of keyword scoring, this paper uses synonym discrimination method to identify the synonyms in the examinees' answer keywords. At the sentence level, attention LSTM model is used to analyze examinees' translation of sentence general idea. Finally, spoken fluency is scored based on tempo/rate and speech distribution. The final translation quality score is obtained by combining the weighted scores of the three parameters. The experimental results show that the proposed method is in good agreement with the result of manual grading, and achieves the expected design goal compared with other methods.

show abstract

A Trait-based Deep Learning Automated Essay Scoring System with Adaptive Feedback

Cited by 13 publications

References 0 publications

Automated Cross-prompt Scoring of Essay Traits

Automated Cross-prompt Scoring of Essay Traits

A review of deep-neural automated essay scoring models

An automatic scoring method for Chinese-English spoken translation based on attention LSTM

Contact Info

Product

Resources

About