Using Latent Semantic Analysis for automated grading programming assignments

Abstract

-Traditionally, computer programming assignments are graded manually by educators. As this task is tedious, timeconsuming and prone to bias, the need for automated grading tool is necessary to reduce the educators' burden and avoid inconsistency and favoritism. Recent researches have claimed that Latent Semantic Analysis (LSA) has the ability to represent human cognitive knowledge to assess essays, retrieving information, classification of documents and indexing. In this paper, we adapt LSA technique to grade computer programming assignments and observe how far it can be applied as an alternative approach to traditional grading methods by human. The grades of the assignments are generated from the cosine similarity that shows how close students' assignments to the model answers in the latent semantic vector space. The results show that LSA is not able to detect orders of computer programming and symbols; however, LSA is able to grade assignments faster and consistently, which avoid bias and reduces the time spent by human.

Continuous assessment in a computer programming course supported by a software tool

“…While syntactic correctness is verified by the compiler/language interpreter, normally the other aspects are assessed manually. This may lead to problems such as: Difficulty: assessment is wasteful and error‐prone. Lack of objectivity: the grade assigned to a specific exercise may differ depending on the evaluator due to factors like tiredness, favoritism, or inconsistencies . Ignoring the building process for the solution: generally, the assessment only takes into account the final result of the students’ source code (their best version), and it does not consider its building process. …”

Pre-Processed Latent Semantic Analysis for Automatic Essay Grading

“…Figure 1 will show the methodology used in this paper. As the general steps in LSA [5] [7], the text will go through stop words removal, building the term matrix and calculating the similarity score between two texts using cosine similarity.…”

“…LSA is a basic method that analyzes the texts to extract its semantic meaning by using support vector machine (SVM) and checked their similarities between the two texts using cosine similarity [5][7] [8]. In LSA approach every term found in the text, sentences or documents will be mapped into the term matrix.…”

“…The meaning of a text will be measured in statistic way based on the relationship among terms in the matrix [9]. LSA does not extract the meaning of the text from the sequence of the words, so that LSA cannot be applied to extract information from text, which sequence and order are important, like programming code and grammatical essay [5] [9].…”

“…Zen [5] used Latent Semantic Analysis (LSA) for grading computer programming assignment and found that LSA can grade essay consistently and faster than manual grading by human, but LSA is lack in detecting the order of commands and symbols in the program. Meanwhile, He et.al.…”