A revised application of cognitive presence automatic classifiers for MOOCs: a new set of indicators revealed?

Abstract

Automatic analysis of the myriad discussion messages in large online courses can support effective educator-learner interaction at scale. Robust classifiers are an essential foundation for the use of automatic analysis of cognitive presence in practice. This study reports on the application of a revised machine learning approach, which was originally developed from traditional, small-scale, for-credit, online courses, to automatically identify the phases of cognitive presence in the discussions from a Philosophy Massive Open Online Course (MOOC). The classifier performed slightly better on the MOOC discussions than similar previous studies have found. A new set of indicators to identify cognitive presence was revealed in the MOOC discussions, unlike those in the traditional courses. This study also cross-validated the classifier using MOOC discussion data from three other disciplines: Medicine, Education, and Humanities. Our results suggest that the cognitive classifier trained using MOOC data in only one discipline cannot yet be applied to other disciplines with sufficient accuracy.

Towards more replicable content analysis for learning analytics

“…Attempts to reproduce existing LA-based results in new contexts have tended to fail more often than they succeed (see e.g. Farrow et al [12], Gardner et al [17], Hu et al [21]). This suggests that tools based upon these methods are not currently robust enough to be used in authentic learning scenarios.…”

Learning Analytics of Totemm Virtual Learning Resources -Transforming Transnational Intercultural Sensitivity for Midwifery Students Through an Inclusive Mobility Model

“…While intention of use of learning analytics has been discussed, there is little evidence that shows improvement of learning outcomes and support of teaching and learning, but there are indications for a shift towards "deeper understanding of students' learning experiences" [8]. Different attempts to reproduce learning analytics findings in new contexts have tend to fail, meaning tools and methods used are not very robust [9]. However, work towards unified frameworks or taxonomies to understand students' experiences and digital content development and interpretation have started to be shaped [10], [11].…”