An Automated Grader for Chinese Essay Combining Shallow and Deep Semantic Attributes

Abstract

Writing is a pivotal part of the language exam, which is considered as a useful tool to accurately reflect students' language competence. As Chinese language tests become popular, manual grading becomes a heavy and expensive task for language test organizers. In the past years, there is a large volume of research about the automated English evaluation systems. Nevertheless, since the Chinese text has more complex grammar and structure, much fewer studies have been investigated on automated Chinese evaluation systems. In this paper, we propose an automated Chinese essay evaluation system called AGCE (Automated Grader for Chinese Essay), which combines shallow and deep semantic attributes of essays. We implement and train our AGCE system on a Chinese essay dataset, which is created by ourselves based on more than 1000 student essays from a Chinese primary school. Experimental results indicate that our AGCE system achieves the quadratic weighted Kappa of 0.7590 on a small dataset, which is of higher grading accuracy compared with other four popular neural network methods trained on large-scale datasets. In addition, our AGCE system can provide constructive feedback about Chinese writing, such as misspelling feedback and grammatical feedback about writers' essays, which is helpful to improve their writing capability. INDEX TERMS Automated Chinese essay evaluation, AGCE, natural language processing, semantic attributes, semantic feedback.

Penerapan Algoritma TF-IDF dan Cosine Similarity untuk Query Pencarian Pada Dataset Destinasi Wisata

“…Selain itu, ukuran kesamaan diakui sebagai fungsi yang mengevaluasi tingkat kemiripan antara sepasang objek teks. Secara ringkas, tingkat similarity mencerminkan intensitas hubungan antara dua titik data [14]. Cosine Similarity adalah ukuran kesamaan yang sering digunakan.…”

Integration of Prediction Scores From Various Automated Essay Scoring Models Using Item Response Theory

“…EASE uses several types of features, including length-based features, partof-speech-based features, prompt-relevant features, and bagof-words-based features. There are many other models that incorporate various types of features, such as word topicality [17], bag-of-super-word embedding [18], argument features (e.g., number of claims and number of supporting relations) created by argument-mining techniques [19], a sentence semantic similarity defined using a graph-based text analysis method [20], and semantic features that are specific to the Chinese language [21].…”

Neural automated writing evaluation for Korean L2 writing

“…The main idea behind MLbased AWE is applying deep learning techniques for automated essay scoring. To compute the score of writing in terms of machine learning, the system has to learn from a training dataset T that comprises a pair of essays x i and scores y i , where (x i , y i ) ∈ T. In the deep learning-based AWE such as in Yang et al (2019), the sequence of words from the essay x i is represented as a sequence of vector representations (i.e., word embeddings). Therefore, the essay x i is composed of m words such that x = (w i,1 , • • • , w i,m ), and the system creates a set of sequences of word embeddings e w i,1 , • • • , e w i,m .…”

“…Machine learning (ML)-based grammar checking (Soni and Thakur 2018) and AWE (Persing, Davis, and Ng 2010; Taghipour and Ng 2016; Yang, Xia, and Zhao 2019) have been proposed and widely used in recent years because of their outstanding performance. The main idea behind ML-based AWE is applying deep learning techniques for automated essay scoring.…”

“…In the deep learning-based AWE such as in Yang et al . (2019), the sequence of words from the essay is represented as a sequence of vector representations (i.e., word embeddings). Therefore, the essay is composed of m words such that , and the system creates a set of sequences of word embeddings .…”