Analytics of Learning Strategies: Role of Course Design and Delivery Modality

Abstract

Generalizability of the value of methods based on learning analytics remains one of the big challenges in the field of learning analytics. One approach to testing generalizability of a method is to apply it consistently in different learning contexts. This study extends a previously published work by examining the generalizability of a learning analytics method proposed for detecting learning tactics and strategies from trace data. The method was applied to the datasets collected in three different course designs and delivery modalities, including flipped classroom, blended learning, and massive open online course. The proposed method combines process mining and sequence analysis. The detected learning strategies are explored in terms of their association with academic performance. The results indicate the applicability of the proposed method across different learning contexts. Moreover, the findings contribute to the understanding of the learning tactics and strategies identified in the trace data: learning tactics proved to be responsive to the course design, whereas learning strategies were found to be more sensitive to the delivery modalities than to the course design. These findings, well aligned with self-regulated learning theory, highlight the association of learning contexts with the choice of learning tactics and strategies. Notes for Practice• Explorations of the detected learning tactics and strategies need to consider both sequential and temporal characteristics.• Learning tactics and strategies are context dependent; therefore, specific learning tactics and strategies have to be interpreted in the particular learning context from which the data originate.• Detected learning tactics should reflect the instructional course design.

A learning analytic approach to unveiling self-regulatory processes in learning tactics

“…The tactics detected were defined as Reading with Quiz Tactic, Assessment and Interaction Tactic, Short Login and Interact Tactic, and Focus on Quiz Tactic. As consistent with the previous research in the area [5,18,20,21], the four learning tactics were reflective of study opportunities provided by the learning context and course design. For instance, the RWQ-Tactic was closely related to the pedagogical intents behind the learning design of the pre-class tasks.…”

“…That is, learners used this tactic to engage with the reading materials and assess or evaluate their knowledge by attempting the quizzes. This tactic aligned with tactics detected in previous studies which were also conducted in blended learning settings with similar course design; for example, Practice and Lecture-oriented Tactics and Lecture and Practice-oriented Tactics in [18] or Diverse Tactic and Multiple-Choice Questions (MCQ) Oriented Tactic in [21]. In addition to the RWQ-Tactic, as the main tactic used by the learners, other tactics such as the AAI-Tactic and FOQ-Tactic were also aligned with findings from previous studies.…”

“…It has also been shown that tactics and strategies identified from learning traces can be used to evaluate the effect of learning analytics-based instructional interventions such as automated personalised learner-directed feedback [20,34]. Furthermore, Matcha and colleagues [21] have demonstrated that learning tactics are generally associated with the opportunities offered by the course design, whereas strategies tend to be the same or similar across courses. While the above cited research efforts have made a notable contribution towards identifying and understanding learning tactics and strategies in various blended and online learning settings, there is a need to deepen our understanding of the identified tactics and strategies; that is, to understand the underlying learning processes that get activated during the enactment of distinct learning tactics and strategies.…”

“…Considering that learning tactics tend to reflect particularities of a course design [21], it would be highly important to identify learning mechanisms that get activated when learners are employing a particular learning tactic (to complete a task defined by the course design). This will enable us to establish associations between learning mechanisms and particular elements of the course design.…”

Students’ sense-making of personalised feedback based on learning analytics

“…In considering students' sense-making of their feedback, it is important to take into account the learning context. The curriculum design plays a significant role in students' SRL and, accordingly, their academic performance (Matcha, Gašević, et al, 2020). However, to date research is limited in exploring how students make sense of and enact their feedback, in different contexts.…”