An Early Warning System to Detect At-Risk Students in Online Higher Education

Abstract

Artificial intelligence has impacted education in recent years. Datafication of education has allowed developing automated methods to detect patterns in extensive collections of educational data to estimate unknown information and behavior about the students. This research has focused on finding accurate predictive models to identify at-risk students. This challenge may reduce the students’ risk of failure or disengage by decreasing the time lag between identification and the real at-risk state. The contribution of this paper is threefold. First, an in-depth analysis of a predictive model to detect at-risk students is performed. This model has been tested using data available in an institutional data mart where curated data from six semesters are available, and a method to obtain the best classifier and training set is proposed. Second, a method to determine a threshold for evaluating the quality of the predictive model is established. Third, an early warning system has been developed and tested in a real educational setting being accurate and useful for its purpose to detect at-risk students in online higher education. The stakeholders (i.e., students and teachers) can analyze the information through different dashboards, and teachers can also send early feedback as an intervention mechanism to mitigate at-risk situations. The system has been evaluated on two undergraduate courses where results shown a high accuracy to correctly detect at-risk students.

Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses

“…Moreover, the analysis of the accuracy of the predictive model embedded into the LIS system (i.e., the GAR model) and the issued predictions shows the capacity of the LIS system to detect the potential at-risk learners effectively from the early stages in both courses. A more detailed discussion about the GAR model accuracy and its comparison with other predictive models can be found in Baneres et al 2020. The results derived from the multivariate analysis are aligned with the results provided by the risk level classification analysis.…”

“…Other systems additionally provide information to learners (Hu et al 2014;Ortigosa et al 2019) since it is essential to inform and empower each stakeholder group. The LIS system (Baneres et al 2020) provides both features by informing learners and teachers about the risk. The system also provides capabilities to easily define, explore, and select custom predictive models based on the available data.…”

“…Following this rationale, the LIS system was developed and tested in the 2020 spring semester in two first-year courses to analyze the impact on learners' engagement and motivation. A full description of the technical architecture can be found on Baneres et al (2021), while capabilities and the model described in the next section can be found on Baneres et al (2020). Thus, this paper aims to analyze the impact on the learners' performance when using the LIS system in the courses they are enrolled in and in the learners' perception about the usefulness and effectiveness of LIS utilization.…”

“…The EWS runs a predictive model called Gradual At-Risk model (abbreviated as GAR model) tailored for each course (Baneres et al 2020). The GAR model is trained before the course starts using historical anonymized data of learners (available at the data mart) that have enrolled in past semesters and likely share characteristics with future learners.…”

“…The minimum mark is obtained by computing the prediction using the ALA's submodel for all possible marks of the corresponding ALA. The mark that makes the prediction change from failing to pass is identified as the minimum mark (Baneres et al 2020). The risk level is updated on each ALA whether the learner's mark is higher or lower than such minimum mark by following the decision tree of Fig.…”