Optimize the Combination of Categorical Variable Encoding and Deep Learning Technique for the Problem of Prediction of Vietnamese Student Academic Performance

Abstract

Deep learning techniques have been successfully applied in many technical fields such as computer vision and natural language processing, and recently researchers have paid much attention to the application of this technology in socioeconomic problems including the student academic performance prediction (SAPP) problem. In this specialization, this study focusses on both designing an appropriate Deep learning model and handling categorical input variables. In fact, categorical data variables are quite popular in student academic performance prediction problem, and deep learning technique in particular or artificial neural network in general only work well with numerical data variables. Therefore, this study investigates the performance of the combination categorical encoding methods including label encoding, one-hot encoding and "learned" embedding encoding with deep learning techniques including Deep Dense neural network and Long short-term memory neural network for SAPP problem. In experiment, this study compared these proposed models with each other and with some prediction methods based on other machine learning algorithms at the same time. The results showed that the categorical data transformation method using the "learned" embedding encoding improved performance of the deep learning models, and its combination with long short-term memory network gave an outstanding result for the researched problem.

Deep Learning Techniques for Predicting Student’s Academic Performance on Virtual Learning Environments: A Review

“…Table 7, shows a summary of all the studies that have used hybrid DL techniques. The analysis of categorical variable transformation techniques and their compatibility with DL models are the main objectives of this study Hien et al (2020). To predict student academic performance, this study compared the effectiveness of label encoding, one-hot encoding, and "learned" embedding encoding with deep learning methods, such as DNNs and LSTM.…”

“…Other architectures include combining DNNs with CNN, where CNN is designed to capture spatial features in input data such as student demographic or academic records, which can then be fed into DNNs to model the temporal dependencies in the data, Sikder et al (2022) applied DNN-CNN and achieved a high accuracy of about 97%. Furthermore, combining DNNs with either LSTM Hien et al (2020); Prabowo et al (2021) or GRU Liu et al (2022c) can be an effective method for predicting student performance. For the DNNs/GRU hybrid model, GRU is used to model the temporal dependencies by extracting the temporal behavior data from the learning Fig.…”

Anomaly Detection in Hospital Claims Using K-Means and Linear Regression

“…Three common methods used are: 1) Label Encoding; 2) One-hot Encoding and its modification; 3) "Learned" Embedding encoding. In the Label encoding method each label of a categorical data variable is assigned to a most suitable integer number [10].…”

“…Where the variance of y is assumed to be response variable and the coefficients of b1 and b0 are called the regression coefficients and x is the predictor variable [10]. Each cluster that has been formed will then look for a straight line equation with a linear regression model as a basis for detecting anomaly data [18].…”