Modeling Students’ Academic Performance Based on Their Interactions with the Online Learning Environment

Abstract

ABSTRACT. The aim of this study is to model students' academic performance based on their interactions in an online learning environment. The dataset includes 10 input attributes extracted from students' learning interaction data. As an output (class) variable, the final grades obtained from their Computer Hardware course were used. The modeling performance of three different classification algorithms were tested (naïve Bayes classifier, classification tree and CN2 rules) on the dataset. All analyses were performed using the Orange data mining tool, and the models were evaluated using ten-fold cross-validation. The results of analysis were presented as a confusion matrix, a decision tree, and if-then rules. The predictive performance of the algorithms was also tested and compared using the classification accuracy (CA), and area under the ROC Curve (AUC) metrics. The experimental results indicate that the naïve Bayes algorithm outperforms other classification algorithms when compared using the CA and AUC metrics. The naïve Bayes algorithm correctly classified 75.4% of the students according to their grade for the course (Fail, Pass, and Good). The classification model also accurately predicted 81.5% of the students who failed, and 91.8% of the students who passed the course. On the other hand, the classification tree and the CN2 algorithms generated models which can be used with confidence in decision making processes by non-expert data mining users.

Identification of Factors Associated With School Effectiveness With Data Mining Techniques: Testing a New Approach

“…Taking into account that in this study decision trees with a very small size (maximum 30 branches) were selected, the levels of average accuracy achieved, which were above 0.76 in the training sample, were satisfactory. Other previous works that did not limit the size of the decision trees achieved overall accuracy levels of between 0.7 and 0.8 (Oskouei and Askari, 2014; Akçapinar et al, 2015; Costa et al, 2017; Kılıç et al, 2017). The study by Liu and Whitford (2011) is particularly interesting since they developed a predictive model of performance from the data of students in PISA 2006 using the C4.5 algorithm.…”

“…Despite the potential that these statistical techniques may hold, their use in the establishment of performance prediction models in compulsory education is sporadic (Hung et al, 2012; Oskouei and Askari, 2014; S̨ara et al, 2015), and their use for the exploration of large-scale assessments is extremely limited (Liu and Ruiz, 2008; Liu and Whitford, 2011; Kılıç et al, 2017; Asensio et al, 2018). A significant presence of EDM can, however, be observed in the study of performance in university education (Guruler et al, 2010; Kasih et al, 2013; Romero et al, 2013; Kirby and Dempster, 2014; Akçapinar et al, 2015; Tan and Shao, 2015; Asif et al, 2017; Casey and Azcona, 2017; Costa et al, 2017).…”

“…As for the statistical technique applied, although numerous works are carried out with the aim of comparing classification algorithms (Jamain and Hand, 2008; Oskouei and Askari, 2014; Yu et al, 2014; Akçapinar et al, 2015; Asif et al, 2017; Costa et al, 2017; Kılıç et al, 2017), there is no agreement in the scientific community on which ones are more appropriate for the prediction of academic performance. “The literature review suggests that in general there is no single classifier that works best in all contexts to provide good prediction” (Kirby and Dempster, 2014).…”

“…The naive Bayes classifier and the EM clustering algorithm achieved the best levels of fit, above C4.5. Finally, a study by Akçapinar et al (2015) analyzed the predictive power of interactions in online environments relating to student performance. Again, the naive Bayes classifier achieved the best levels of fit, slightly higher than those obtained with decision tree C4.5.…”

Learning Analytics: a Metacognitive Tool to Engage Students

“…For that reason, it is rather important to take a closer look at various data analysis techniques applying learning analytics. A note should be made that the most commonly used practices for data analysis employed by learning analytics include the following data analysis practices: statistics, information visualization, educational data mining, and social network analysis (Chatti et al, 2012;Akçapinar et al, 2015;Algarni, 2016). These methods have appeared to be useful in combining students into groups based on their similarities, providing suggestions about course materials, and making predictions about students' behaviour (Akçapinar et al, 2015), which has eventually led to improvement of the whole learning experience.…”