A generalized classifier to identify online learning tool disengagement at scale

Abstract

Student success, a major focus in higher education, in part, requires students to remain actively engaged in the required coursework. Identifying student disengagement, when a student stops completing coursework, at scale has been a continuing challenge for higher education due to the heterogeneity of traditional college courses. This research uses data from Connect by McGraw-Hill Education, a widely used online learning tool, to build a classifier to identify learning tool disengagement at scale. This classifier was trained and tested on four years of historical data, representing 4.5 million students in 175,000 courses, across 256 disciplines. Results show that the classifier is effective in identifying disengagement within the online learning tool against baselines, across time, and within and across disciplines. The classifier was also effective in identifying students at risk of disengaging from Connect and then earning unsuccessful grades in a pilot course for which the assignments in Connect were worth a relatively small portion of the overall course grade. Because Connect is widely used, this classifier is positioned to be a good tool for instructors and institutions to identify students at risk for disengagement from coursework. Instructors and institutions can use this information to design and implement interventions to improve engagement and improve student success at the institution in key courses.

Incorporating an LMS Learning Analytic into Proactive Advising: Validity and Use in a Randomized Experiment

“…Several LMS analytics have been validated for predicting student outcomes. Feild et al (2018) examined 4.5 million students' LMS logs in 175,000 courses across 10 disciplines using the McGraw Hill Connect platform, connected to McGraw Hill's electronic text enterprise. They developed a classifier focused on ten LMS features, including assignment submission rates and time on assignments, resulting in a "disengagement classification score" that identified 93% of students who ultimately failed or withdrew from courses.…”

Virtual Agents for Real Teachers: Applying AI to Support Professional Development of Proportional Reasoning

“…Research on the impact of virtual agents on engagement and motivation is also relatively strong, but requires further investigation for teachers as learners. Overall, disengagement is often a problem for online learning (Feild et al 2018) and virtual agents have shown the ability to increase motivation in computer-based learning (Sträfling et al 2010). However, research has not studied how teachers specifically react to pedagogical agents and research with other adult professionals limited.…”