Open Problems in Indonesian Automatic Essay Scoring System

Rahutomo, Faisal; Roshinta, Trisna Ari; Rohadi, Erfan; Siradjuddin, Indrazno; Ariyanto, Rudy; Setiawan, Awan; Adhisuwignjo, Supriatna

doi:10.14419/ijet.v7i4.44.26974

Cited by 8 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Sebagai hasilnya, esai dianggap sebagai alat evaluasi yang efektif untuk mengukur pemahaman siswa terhadap materi yang dipelajari sebelumnya [2].…”

Section: Pendahuluanunclassified

Implementasi Long-Short Term Memory (Lstm) Untuk Generasi Feedback Berbahasa Indonesia Pada Sistem Penilaian Esai

Lanasemba

2024

View full text Add to dashboard Cite

Esai digunakan sebagai alat evaluasi yang memerlukan respon subjektif dan mendalam dari siswa, menyoroti kemampuan kognitif yang tinggi dan keterampilan menulis yang baik. Pentingnya umpan balik dalam mengidentifikasi aspek-aspek yang perlu ditingkatkan dalam jawaban siswa telah diakui secara luas. Automated Essay Scoring (AES) adalah sistem yang menggunakan teknologi machine learning untuk mengevaluasi esai siswa secara otomatis. Meskipun telah ada penelitian tentang AES berbahasa Indonesia seperti SIMPLE, Automated Bahasa Indonesia Essay Evaluation with Latent Semantic Analysis, dan E-learning Automated Essay Scoring System Menggunakan Metode Searching Text Similarity Matching Text, namun implementasi umpan balik untuk AES berbahasa Indonesia masih belum sepenuhnya dilakukan. Memberikan umpan balik terhadap setiap esai bisa menjadi pekerjaan yang memakan waktu, terutama jika terdapat banyak esai yang perlu dinilai. Ini bisa menyebabkan kelelahan bagi pengajar dan berisiko mengurangi kualitas respon yang diberikan. Long-Short Term Memory (LSTM) memiliki potensi sebagai sistem umpan balik dalam bahasa Indonesia. Sebagai salah satu jenis algoritma dalam keluarga Recurrent Neural Network (RNN), LSTM unggul dalam mengatasi struktur data sekuensial yang kompleks, seperti teks esai, dan mampu mengidentifikasi pola serta hubungan antar kata-kata dalam teks secara akurat. Oleh karena itu, penelitian ini bertujuan untuk mengimplementasikan sistem umpan balik pada AES berbahasa Indonesia dengan menggunakan algoritma LSTM. Implementasi LSTM kemudian dilakukan untuk pengembangan umpan balik dengan output berupa kalimat umpan balik berjenis positif dan negatif dan AES dengan output berupa skor dengan range satu hingga empat. Model umpan balik kemudian diimplementasikan dengan AES sehingga skor yang dikeluarkan oleh AES berpengaruh terhadap jenis kalimat umpan balik yang diberikan.

show abstract

“…Sebagai hasilnya, esai dianggap sebagai alat evaluasi yang efektif untuk mengukur pemahaman siswa terhadap materi yang dipelajari sebelumnya [2].…”

Section: Pendahuluanunclassified

Implementasi Long-Short Term Memory (Lstm) Untuk Generasi Feedback Berbahasa Indonesia Pada Sistem Penilaian Esai

Lanasemba

2024

View full text Add to dashboard Cite

show abstract

“…Based on these studies, we proposed an automated essay scoring in the dataset [16,17] using the semantic features of sentence embeddings SBERT with sequence length handling by averaging as in the previous study [8]. The sentence embeddings transform text answers into vectors as features that can be processed further.…”

Section: Lstm-based Automated Scoringmentioning

confidence: 99%

Evaluating the Impact of Sentence Tokenization on Indonesian Automated Essay Scoring Using Pretrained Sentence Embeddings

Chamidah,

Yulianti,

Budi

2023

RIA

View full text Add to dashboard Cite

Automated Essay Scoring (AES) systems are designed to expedite the assessment process, where human scoring is frequently slow and subject to inconsistencies and inaccuracies. This study, therefore, investigates the role of sentence tokenization in the performance of Indonesian Automated Essay Scoring, given that Natural Language Processing (NLP) techniques are requisite in AES to handle student responses that present identical semantic meanings but vary in length. A distinct approach was adopted in which full answers were not vectorized; instead, they were fragmented into sentences prior to vectorization. This method was deemed potentially more effective due to the high probability of discrepancies in sentence order between reference and student responses. Sentence embeddings, which encapsulate a sentence as a sole vector, were utilized. Pretrained SBERT-based sentence embeddings were employed to vectorize sentences from both reference answers and student responses, serving as semantic features for the Siamese Manhattan LSTM (MaLSTM) model. The MaLSTM model possesses the ability to process two inputs and evaluate their similarity using the Manhattan distance metric and use this similarity value as a predictive scoring output. This score was subsequently compared to human scores using the Root Mean Square Error (RMSE) and Pearson Correlation. Interestingly, sentence embeddings without tokenization slightly outperformed those with sentence splitting, as evidenced by a 0.61% improvement in RMSE and a 0.01 increase in Pearson Correlation. The results obtained indicate that sentence tokenization, as applied to the Indonesian Automated Essay Scoring dataset, does not have a notable impact on essay scoring performance. Therefore, it may be concluded that the application of sentence tokenization is not a necessary step in this dataset's text-processing phase of AES.

show abstract

“…For the data set preparation, we used two data sets: a ready-to-use data set, which will further be referred to as the Rahutomo data set, and an own-built data set, which will further be referred to as the basic programming data set. Rahutomo data set consists of 40 pairs of questions and answers divided into four categories: politics, lifestyle, sport, and technology [9]. Labels were later converted to decimals with a scale of 0 to 5 to facilitate comparison to previous studies.…”

Section: Data Set Preparationmentioning

confidence: 99%

“…The statistic report on the number of students and educators in Indonesian schools and universities [10]- [14] Just like the rarity of related studies, the Indonesian ASAG data set was also hard to obtain. One of the studies that provided an open Indonesian ASAG data set was [9]. However, some of the data set's characteristics were not suitable for our research, such as the number of responses to each question being too few and the sentences being too long.…”

Section: Introductionmentioning

confidence: 99%

Indonesian automatic short answer grading system

Salim

Pratamaputra

et al. 2022

Bulletin EEI

View full text Add to dashboard Cite

Short answer question is one of the methods used to evaluate student cognitive abilities, including memorizing, designing, and freely expressing answers based on their thoughts. Unfortunately, grading short answers is more complicated than grading multiple choices answers. For that problem, several studies have tried to build an artificial intelligence system called automatic short answer grading (ASAG). We tried to improve the accuracy of the ASAG system at scoring student answers in Indonesian by enhancing the earlier state-of-the-art models and methods. They were the bidirectional encoder representations from transformer (BERT) with fine-tuning approach and ridge regression models utilizing advanced feature extraction. We conducted this study by doing stages of literature review, data set preparation, model development, implementation, and comparison. Using two different ASAG data sets, the best result of this study was an achievement of 0.9508 in pearson’s correlation and 0.4138 in root-mean-square error (RMSE) by the BERT-based model with the fine-tuning approach. This result outperformed the results of the previous studies using the same evaluation metrics. Thus, it proved our ASAG system using the BERT model with fine-tuning approach can improve the accuracy of grading short answers.

show abstract

Open Problems in Indonesian Automatic Essay Scoring System

Cited by 8 publications

References 12 publications

Implementasi Long-Short Term Memory (Lstm) Untuk Generasi Feedback Berbahasa Indonesia Pada Sistem Penilaian Esai

Implementasi Long-Short Term Memory (Lstm) Untuk Generasi Feedback Berbahasa Indonesia Pada Sistem Penilaian Esai

Evaluating the Impact of Sentence Tokenization on Indonesian Automated Essay Scoring Using Pretrained Sentence Embeddings

Indonesian automatic short answer grading system

Contact Info

Product

Resources

About