Developing an early-warning system for spotting at-risk students by using eBook interaction logs

Abstract

Early prediction systems have already been applied successfully in various educational contexts. In this study, we investigated developing an early prediction system in the context of eBook-based teaching-learning and used students' eBook reading data to develop an early warning system for students at-risk of academic failure-students whose academic performance is low. To determine the best performing model and optimum time for possible interventions we created prediction models by using 13 prediction algorithms with the data from different weeks of the course. We also tested effects of data transformation on prediction models. 10-fold cross-validation was used for all prediction models. Accuracy and Kappa metrics were used to compare the performance of the models. Our results revealed that in a sixteen-week long course all models reached their highest performance with the data from the 15th week. On the other hand, starting from the 3rd week, the models classified low and high performing students with an accuracy of over 79%. In terms of algorithms, Random Forest (RF) outperformed other algorithms when raw data were used, however, with the transformed data J48 algorithm performed better. When categorical data were used, Naive Bayes (NB) outperformed other algorithms. Results also indicated that models with transformed data performed lower than the models created using categorical data. However, models with categorical data showed similar performance with models with raw data. The implications of the results presented in this research were also discussed with respect to the field of Learning Analytics.

Using learning analytics to develop early-warning system for at-risk students

“…On the other hand, the regular logging-on of students in the setting and the time spent on it emerged as significant features. Akçapınar, Hasnine, Majumdar, Flanagan, and Ogata (2019), indicated the impact of logging onto learning settings regularly on academic success rather than carrying out many activities in one session. In a comparison of the performances of classification algorithms developed using the complete data set, the highest level of classification accuracy was reached using the kNN and CN2 algorithms and under the condition in which data was converted into categorical form using the equal width method and important features selected according to the Gini index.…”

Systematic Literature Review of Predictive Analysis Tools in Higher Education

“…Additionally, one importance characteristic of many of these studies is their experimental nature, leading them to try out and compare the performance of many different algorithms. Examples of this trend are Akcapinar et al [36], who tested 13 different classification algorithms, and Adekitan and Salau [18], who tried both classifiers and regression algorithms in their work. One of the most important defining characteristics of each predictive application was the types of data that it used as input.…”

“…Two studies stand out above the rest in terms of population size: Ornelas and Ordonez had data of 8658 students belonging to 13 different courses [10], while Waddington and Nam collected data regarding 8762 students across 10 semesters [27]. On the other hand, the system presented by Chen was tested on a class of only 38 students [21], while Akcapinar et al included only 90 students in their study [36], which can be regarded as insufficient in order to make a solid evaluation of a predictor's performance.…”

“…Akcapinar et al built an EWS intended to work together with BookRoll, an e-book management system used in several Asian universities that provides access to course materials [36]. This tool collected student interactions with BookRoll as input, in the form of Experience API (xAPI) statements.…”

Predicting at-Risk Students at Different Percentages of Course Length for Early Intervention Using Machine Learning Models

“…Akçapınar Gökhan developed an early warning system that used students' eBook reading data to predict students atrisk of academic failure [40]. To develop the best predictive model, 13 ML algorithms were used to train the model using data from different weeks of the semester.…”