Using machine learning techniques to predict academic success in an introductory programming course

Predicting Students’ Outcome in an Introductory Programming Course: Leveraging the Student Background

“…It is also interesting to note that the best model for all students (SVMs) achieved an accuracy of 67.71% with only two predictors, which is slightly below the best result obtained in previous work, where the best accuracy was 68.6% considering 21 predictors [27]. In this case, the same model exhibits better specificity (72.24%) than sensitivity (63.21%).…”

“…The goal of this study is to establish which students' features best predict students' outcomes in FCYPs' theory part. For this purpose, the data set considers the variables described in Table 1, identified as relevant in previous work [44] for being the ones that best predict student dropout by the end of the first semester and in [27] for predicting whether students pass or fail the theory component of the course using the same data set. It should be noted that the Chilean grading scale applies to final grades for the following variables: AL, PHY, ALG, MET, INT, and GPA.…”

“…Their findings show that features related to self-efficacy and the importance they place on programming skills have the greatest predictive potential. In a different vein, the preliminary study on which this work is based [27] attempted to identify those previously registered variables with the highest correlation with a passing grade in an introductory programming course imparted in the second semester of 20 engineering programmes. For this purpose, three different data sets of students' features were considered.…”

“…The structure of the research presented in this paper is summarised in Figure 1 and further described in following sections. This work is an extension of a preliminary conference paper presented in [27], which attempted to identify those characteristics with the highest correlation with a passing grade in an introductory programming course using the same variable sets. The extensions in this work include (1) the use of statistical methods to remove correlated variables; (2) the exclusion of Machine Learning algorithms that did not yield satisfactory results in the previous study (namely CART Trees and Extreme Learning Machines); (3) training models with subsets of data based on programme duration and the Department of the Faculty of Engineering in charge of each programme; and (4) based on the aforementioned extensions, a new analysis of the results and discussion.…”

“…The following Machine Learning techniques were selected to build classifiers on the selected features: radial kernel SVMs [46], MLR [47], RFs [48], and XGB [49]. These methods were chosen because of their performance in previous work [27]. Each model was tuned considering 10 repeats of a 10-fold cross-validation (for each repeat, observations are separated in 10 groups or folds, and then, for each fold, a model is tuned using the remaining 9 folds as a training set and the current one to assess the model's performance).…”